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FOREWORD 


Tue present publication represents the final Report of the 
Scottish Committee, or Delegation, in the International Examina- 
tion Inquiry initiated by the Carnegie Corporation of America, 
the Carnegie Foundation and the International Institute ot 
Teachers College, Columbia University, and financed jointly by 
the Carnegie Corporation and the Carnegie Foundation. It 
also represents the completion of the tasks undertaken in 1931 at 
the Eastbourne Conference by the Scottish Delegation. ‘This, the 
fourth of our major investigations, had not defined itself very 
clearly until just before the second Conference, held at Folkestone 
in 1935, where the plans were submitted for criticism and for 
approval. The third Conference, at Dinard in 1938, saw the 
investigation so far advanced that Professor McClelland was 
able to give a detailed account of what had been completed and 
what still remained to be done. Now, four years later, he 
is able to present to the Committee and to the public the 
full results. It will, I think, be generally agreed that this has 
been the most important and the most onerous of all the investiga- 
tions sponsored by the Committee; it consequently seems incum- 
bent on me as Convener to express our indebtedness to Professor 
McClelland, who has throughout borne on his own shoulders 
practically the whole burden. The measure of that indebtedness 
can be assessed by the reader of this work. 

At the Eastbourne Conference the Chairman, Dr Paul Monroe, 
stressed the sociological implications of examinations, and pointed 
out that these implications constitute the most general aspect of 
the whole series of investigations undertaken by the various 
national delegations. The enormous sociological significance of the 
present investigation has been indicated by Professor McClelland, 
and any elaboration is superfluous. It is worth noting, however, 
that the Mental Survey investigations of 1932 and 1935-37 might 
be said to represent the first stage in a great educational and 
sociological study, of which the present investigation represents 
the second stage, and that a third stage remains to be undertaken— 
a mental and educational survey of the population of our secondary 
and technical schools, our universities and central institutions, as 
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well as our adult education classes. This third stage has already 
been envisaged by the Scottish Council for Research in Education, 
but financial considerations preclude for the time being the 
possibility of undertaking an investigation so complex, so difficult 
and so costly. 

Although the International Examination Inquiry as initiated 
and financed by the International Institute of Teachers College 
has now terminated, at least so far as the Scottish Delegation is 
concerned, we are still faced with this great task, and with the 
study of other problems concerned with examinations. "Тһе hope 
may be expressed that, when the present difficult days are past, 
it will again be possible to secure international collaboration in 
this work. At the closing session of the Dinard Conference in 
1938, on the motion of the Convener of the Scottish Delegation, 
it was unanimously agreed that an international committee for 
the study of examinations be appointed, and that it consist in 
the first instance of the chairmen of the existing committees. 
Subsequently Professor Godfrey H. Thomson consented to act as 
secretary to such a committee. Unfortunately the unsettled 
state of the Continent, and then the War, made it impossible 
to carry the organisation of the committee beyond the skeleton 
stage. When peace returns, it may not be too much to expect 
that this committee will begin to function, and continue the 


work so successfully prosecuted for seven years under the egis 
of the International Instit 


ute, and the inspiring leadership of 
Dr Paul Monroe. : = 


It only remains to express once more in the name of the Scottish 
International Examinations Inquiry Committee and the Scottish 
Council for Research in Education their sense of the deep debt 


they owe—and Scottish education owes—to the International 
Institute of Teachers Coll 


ry ` ege for the continued encouragement 

A ave received, and for the very substantial financial assistance 
without which the investigation could not have been undertaken 
and published. 


JAMES DREVER, 


Convener. 


| 
| 
| 
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PREFACE 


Тиз work is a scientific study of certain problems of selection 
and guidance that arise at the first big sifting of the material in 
an educational system; and, as such, it is not directly concerned 
with many of the deeper issues of educational and examination 
policy. We set out to find answers to a number of definite 
questions in which administrators are interested, but we offer no 
opinion as to whether, in a sound educational system, the questions 
would arise in these particular forms, or would, indeed, arise at all. 

Our thanks are due to the Convener and Members of the 
Dundee Education Committee not merely for their ready and 
Benerous response to our application for facilities to conduct the 
experiment, but also for the encouraging interest which they 
have taken in its progress. To their Director of Education, 
Mr John R. Cameron, М.А., we have been indebted for the 
preparation of the examination papers and for constant guidance 
and help in all our difficulties. Dr A. E. Kidd, M.B., С.М. 
D.P.H., former Chief Medical Officer for Schools, and his staff 
undertook the heavy task of furnishing us with health gradings and 
medical reports for the complete group of over 3,000 pupils; and 
Dr Kidd also prepared the classification of schools on the basis ot 
social class. We have also to record our appreciation of the co- 
operation of the head teachers and staffs of the City schools, to 
whom we are deeply grateful for the friendly and helpful way in 
which they met our heavy demands upon their time and patience. 

Lengthy as the Report is, it is only to those with actual 
experience of similar research that it will convey a full realisation 
of the labour involved. "Тһе determination of a single correlation 
coefficient, when the numbers are large, is a long and laborious 
operation; and, to ensure accuracy, all our calculations were done 
twice, by different teams. With this duplication the number 
of direct correlation calculations in the main Inquiry was over 
3,000: the testing programme involved the correction of over 
20,000 scripts. 

To embark upon such an undertaking without a large full-time 
staff implied a belief, which events have proved to be well-founded, 
that we could safely count upon the help of our students and of 
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our colleagues in the City schools. Тһе Inquiry was, in fact, an 
adventure in large scale co-operative research carried out by 
voluntary part-time assistants; and a brief description of the 
organisation may be of interest to other investigators. . 

My two partners in the general conduct of the Inquiry were 
Miss Margaret Young, М.А., B.Ed., Lecturer in Experimental 
Education in Dundee Training College, and Douglas M. McIntosh, 
M.A., B.Sc., B.Ed., Ph.D., Assistant Director of Education for 
the County of Fife. Miss Young's special responsibilities included 
charge of the testing, preparation of circulars of instructions, 
training teams of testers and correctors, etc. She is the writer 
of Chapter I, in which she gives an account of these matters. 
Dr McIntosh, who has written Chapter II, undertook such tasks 
as the standardisation of the tests, organisation of the correlation 
calculations, training teams of computers, апа зо оп. Yet neither 
a list of special duties nor the occasional references in the Report 
can give an adequate appreciation of what the Inquiry owes to 
these two principal assistants. Each envisaged the research as a 
whole and helped with practically every aspect of it. 

So also did Miss Muriel Mitchell, M.A., who, before her 
appointment to the administrative staff of the College, had charge 
of the rapidly mounting mass of documents in the room set aside 
for the Inquiry. Since 1936 she has given voluntary help of the 
utmost value and has been much in Tequest as a consultant to 
whom student assistants went singly or in groups for advice in 
their statistical difficulties. 

he correction of the tests and the correlation calculations 
were carried out by a veritable army of helpers, over 600 in all, in 
whose ranks were members of the College staff, head teachers 
and teachers of the City schools, and students of the Training 
College. Most of them had to be trained for the work, and this 
training was given in the first place by Miss Young and Dr 
McIntosh. Later, certain of the more skilled workers: acted as 
group leaders and trained new teams. Two of these group 
leaders, Alex. S. Robertson, M.A., and James C. Kidd, M.A., B.A., 


gave particularly valuable help with the more difficult calculations 
and many special problems, 


The teams of correctors and 


à l computers fell roughly into two 
types, the ‘permanent 


and the ‘occasional? Into the former 
came a most faithful group who worked from 6 to 9 p.m. on one 
or two evenings a week for several sessions. In the latter we had 
groups of students who worked in forenoons during vacations; 
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others who worked in free periods during the College session; 
others who worked individualy at home, cóming up to the 
evening meetings periodically to discuss difficulties. 

Finally, about fifty graduate students undertook studies of 
specialaspects. Each year, after a talk on the aims and progress 
of the Inquiry, a list of problems was submitted from which 
interested students could make their choice. "There was no lack 
of volunteers for these investigations, many of the results of which 
will be found in the Report. ; 
. It may not Бе out of place to say a word as to the part played 
by the Inquiry in the life of the Training College. Тһе students 
help was invaluable, and our gratitude to them is not diminished 
by the belief that the benefits were mutual. For many of them 
simple statistics forms part of the course of training, and the 
calculations which they carried out in connection with the 
Inquiry merely replaced class-room exercises which would have 
had less reality and interest. Moreover, while all types of student 
participated, they were not asked to work in a blind mechanical 
manner. Before setting to work they understood the plan of 
the research and the way in which their particular calculation 
fitted into the whole. Whenever possible we let them know the 
results which their group had obtained, and discussed their 
significance. In short, the Inquiry was a central research interest 
for the whole College for six years. Its general headquarters, the 
Inquiry room, was a meeting-place for students from all sections 
who were co-operating in the work. It had a real research 
atmosphere, and no one could associate with the students in it 
without a feeling that the project was an enriching influence in the 
life of the College. 

Of the many colleagues who helped us we are specially indebted 
to Mr B, Babington Smith, M.A., of the Psychology Department 
of the University of St Andrews, with whom we had many 
fruitful discussions on statistical problems, and to the following 
members of the staff of the Dundee Training College: Mr 
Andrew Nairn, Principal Lecturer in Art, who prepared the 
diagrams; Mr Robert B. Martin, Chief Clerk, who acted as 
Treasurer; Miss Dorothy Mess, and other members of the 
administrative staff whose help included the typing and duplicating 
of circulars, etc.; and Mr George Caithness, janitor, who helped 
us in ways too numerous to specify. : 

"Throughout the Inquiry, from the original formulation of the 
plan to the final preparation of the Report, we had the benefit of 
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the expert advice of the Research Council's Director, Dr R. R. 
Rusk. The extent of our indebtedness to him will be best 
appreciated by those who have carried out researches for the 
Council. i 

Finally, a reference may be made to the Report itself. It is 
primarily a research report; and the ways of educational research 
have become very technical: yet we have tried to present it in 
such a way that the results will not be rendered inaccessible to 
the practical administrator through being expressed in the un- 
intelligible mathematical jargon that would be more pleasing to the 
expert. We have kept the mathematics as far in the background 
as possible, and we believe that the lay reader, whose statistical 
attainments are limited to a knowledge of the meaning of the 
terms ‘standard deviation’ and ‘correlation coefficient,’ will be 
able to follow the text. He should simply ignore the footnotes. 
These are for those who are interested in the techniques; but to 
avoid possible disappointment, it should be mentioned that most 
of our methods and formule had to be devised ad boc, and that 
the references often give merely the starting-point from which it 
will be possible to retrace our mathematical steps. 

We have tried to throw a little light on some of the dark places 


_ of the selection problem: but by the time the reader has finished. 


the Report he will realise that the torch has been a heavy one to 
carry—and we now willingly hand it on to others. 
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CHAPTER I 
PLAN AND ORGANISATION OF THE INQUIRY 


Tue QUALIFYING STAGE ім ScorrISH EDUCATION 


Wuar is known as the Qualifying stage in Scottish education is 
that at which a pupil is transferred from the uniform primary 
course to differentiated post-primary courses. The normal асе 
of transfer is 12 years. 8 

Until December 1921 all pupils who had completed a prima 
course had to undergo a Qualifying examination, conducted he 
Н.М. Inspectors, the function of which was to ascertain d 
the pupil had satisfactorily completed the primary stage and was 
fit to enter a post-primary course. When this examination was 
abolished, the Scottish Education Department indicated, in 
Circular No. 44, that the Qualifying stage *must continue to mark 
something of an epoch in the school career of every pupil’; and 
Education Authorities were advised to substitute for the Depart- 
ment's Examination *some arrangement under which those who 
have taught and those who are to teach particular individuals 
shall combine in an endeavour to estimate the potentialities of the 
material to be handled.’ v 

Since 1922 there has been a considerable amount of experiment 
with different methods of complying with this injunction. Each 
Education Committee has evolved its own scheme of promotion 
at the Qualifying stage, and these schemes are modified from time 
to time in the light of experience gained. They range from 
simple systems where only one measure of attainment is used, to 
elaborate systems involving the use of standardised mental and 
scholastic tests, ordinary examinations and teachers’ estimates, 

The uses which are made of the results of the examinations at 


the Qualifying stage vary from Area to Area. 


Tue OBJECT or THE Inourry 


The aim of this Inquiry is to study the problems of selection 
dance that arise at the Qualifying stage, and to find the 


and gui д í : 
er of assessment of ability and attainment which will give the 
® 
I 
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most reliable forecast of the type of post-primary course for which 
each pupil is fitted. Іп particular the object is to find answers 
to the following questions: 


What are the relative predictive values of (2) ordinary examina- 
tions, (b) teachers’ estimates, (с) group intelligence tests and 
(d) standardised scholastic tests? В 

What are the best combinations of the above measures 1n 
selecting pupils for secondary courses? 


Tue GENERAL PLAN or THE INQUIRY 


Satisfactory answers to such questions can be obtained only | 
by means of a large-scale follow-up investigation. We therefore 
first tested the complete Qualifying group in a Scottish City by 
all the methods of assessment at present in use, obtaining additional 
data on special rating cards. 'ТҺе progress of these pupils in the 
various post-primary courses was then followed for three years. 

We proposed to continue the follow-up of the senior secondary 
pupils for five years, but this had to be abandoned because of war 
conditions. 


THE QUALIFYING GROUP 


Тһе Qualifying group consisted of all pupils presented for pro- 
motion to post-primary courses in Session 1935-1936. The total 
number was 3,229, of whom 1,130 were presented in December 
1935 and 2,099 in June 1936. In view of the two dates of 
presentation, duplicate tests and examinations had to be prepared. 
The age distributions of the two groups were as follows: 


A December 1935 June 1936 
> N Per cent. N Per cent. 
14-15 5 3 "8 9 4 
13-14 46 41 126 66 
12-13 578 51-1 1,018 48:5 
11-12 501 44:3 940 44:8 
І0-ІІ | 2 2 6 "3 
1,130 2,099 


THE AGE-GROUPS 


'The age-groups were used for the standardisation of the in- 
telligence and scholastic tests. "They consisted of all pupils in 
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the City of ages from 11 years 6 months to 12 years 5 months on 
the dates of the two examinations. The distributions of these 
аве-втопрѕ among the classes were as follows: 


a December 1935 June 1936 
5 N Per cent. N Per cent. 
Primary 
TI. я я . 48 r6 45 I:5 
ІУ . «7% а 249 8-1 348 II:9 
V (Qualifying class) . 1,860 60-3 1,940 66-7 
$есопдагу 
First year . 4 d 906 294 576 19:8 
Second year " 19 6 2 I 
3,082 2,911 


THE QUALIFYING DATA 
Standardised Tests 
The folowing Moray House Tests prepared by Professor 
Godfrey Н. Thomson were used: 1 


Test Duration in minutes 
Subject ^. 73и Len Practice Test Total 
Intelligence . MHT 20 MHT 21 10 45 55% 
English —. . МЕ 7; МНЕ 8 5% 40 40 
Arithmetic - MHA 7 МНА 8 55 30 30 


Qualifying Examinations 


Subject Description Duration in minutes 
English v - Dictation and Spelling - 30 approximately 
Composition ; = s 2 45 


General English 
(а) Interpretation of passage Р, 
(0) Simple questions on English usage бо 
Arithmetic . + Mena . Е А 5 є ЕС 
Mechanical and problems . * «60; 


\ 


1 Particulars of the Moray House Tests may be found in What are Moray 
House Tests? London: University of London Press, Ltd., 1940. 
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Тһе papers, which were based upon those set in recent years to 
Qualifying pupils in various Scottish counties, were prepared by 
the Director of Education for the City. Copies of the December 
papers are given in Appendix A. 


Teachers. Estimates of Attainment in the Primary School 

The head masters and teachers were in the habit of estimating 
the attainment of each pupil at the Qualifying stage, and these 
estimates were obtained from the schedules sent to the Director 
of Education. Marks were on a percentage basis in English and 
Arithmetic, while a qualitative grading was given in practical 
subjects such as Handwork and Drawing. 
Bursary Examinations 

The City Education Committee allowed the results of the 
рынан. examinations in English апа Arithmetic set to applicants 
for bursaries for senior secondary courses to be used. 


Rating Cards 2 
А rating сага was devised for the purpose of securing qualitative 

estimates of each child's health, ability, attainment and personality. 

Тһе primary head masters were responsible for completing this 


card, guided by a carefully prepared memorandum of instructions. 
The grading for health was supplied by the Chief Medical Officer 


for Schools. This rating card is shown in Appendix В. 


ADMINISTRATION OF THE TESTS AND EXAMINATIONS 


Compilation of a Card Index 

Particulars regarding each pupil were supplied by head masters 
on special cards. "Гһеве cards provided an estimate of numbers 
in each school and particulars for the tabulation sheets which had 


| to be prepared before the tests and examinations were corrected. 


Arrangements in Schools 
Head masters co-operated in securing the best possible testing 


^ conditions. Pupils were generally tested in groups of 25 to 30, 


seated at single desks. Circulars were prepared to ensure uni- 
formity, and in deciding matters relating to internal school 
arrangements the assistance of the Director of Education was 
invaluable. 'The following circulars to head masters were 
issued: 
Circular No. I. Mental and Scholastic Tests. This gave 
instructions regarding time-table, arrangement of pupils in 
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groups, names of examiners and supervisors, material required, 
return of material. 


Circular No. II. Mental and Scholastic Tests. This brief 
memorandum confirmed the times stated in Circular No. I 
and gave full details of the duration of the tests. 


Circulars of a similar type were issued in connection with the 
examinations. 


Time-table of Tests and Examinations 

The intelligence and scholastic tests were given on three con- 
secutive days. "Тһе examinations were given on two consecutive 
days about a fortnight after the last test. " 


Distribution and Collection of Material 

Great care was exercised in the distribution and collection of 
standardised test and examination papers. Sealed packets were 
delivered to each head master and handed by him to an accredited 
supervisor before each test period. The standardised tests had 
not previously been used in the City. All used and unused tests, 
all examination question papers and examination scripts, were 
returned to the Dundee Training College at the close of each 
day's testing. 


Teams of Testers and Supervisors 

The standardised tests were administered by teams of teachers 
and students trained in this technique. The same teams con- 
ducted the Qualifying examinations under carefully controlled 
conditions. Meetings were arranged with teachers and students 
at which details of procedure were discussed, and an accredited 
Supervisor was appointed to each school to make contact with 
the head master. "The following circulars were distributed to the 
testers and supervisors: 


Memorandum for Testers No. І. Mental and Scholastic Tests. 
Details were given of dates and duration of test periods, and 
instructions as to test procedure were carefully stated, 

Memorandum for Testers No. II. Qualifying Examinations. 


Details were given as to distribution of examination material, 
heading of answer sheets, and administration of examinations. 


Memoranda for Supervisors of Tests and Examinations. These 


memoranda gave instructions for the distribution and collec- 
tion of test material, examination material and absentee slips. 
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The care exercised in the conduct of the ordinary type i: 
examinations was occasioned by the belief that such pear 
may be at a disadvantage relative to standardised tests if t el 
are given under ordinary school conditions while the tests а 
administered under rigorous research conditions. 


Correction and Tabulation of Results : E f the 
(а) Standardised tests. The efficient administration © e 
tests and examinations was due to the loyal team work of db 
masters and teachers of primary and secondary schools assiste d 
Training College students. "Their continued co-operation көлү 
light work of the formidable task of correcting the 10,000 standar 
ised tests. The correction, checking, calculation of quotients an 
tabulation of results were carried out in the Training College 9Y 
teams of volunteers under supervision. А sae 
(0) Qualifying examination scripts. The Qualifying ane 
tion scripts were marked by a paid examiner who has for ma 


ac Erga ; tis 
years corrected the Qualifying examination papers in a Scot 
Education Area. 


THE FOLLOW-UP 


The only selection for five-year Secondary education resulted 
from the fact that, at this time, fees were Charged in all the senior 
secondary schools. Pupils desiring a full secondary education, who 
would have been debarred by reason of the expense involved, took 
a special examination. If they attained a certain standard in this 


examination, defined from year to year by the Education Com- 
mittee, they were admitted free to à senior secondary school. 


The results of the standardised tests апд Qualifying examina- 


tions given in this investigation were not used to influence 
promotion. b 
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Tracing tbe Qualifying Group 

The tracing of the 3,229 pupils presented for the Qualifying 
examination in 1035-1936 proved to be an intricate task. Тһе 
courses to which they proceeded are shown below: 


Course т М Рег сепї. 
Senior secondary courses. а Я қ Р 461 143 
Junior secondary courses 
Boys' technical = " А . 5 870 269 
Girls’ technical E E қ " " 763 23:6 
Commercial . ч з $ z . 379 117 
Special classes for backward pupils 5 357 ILI 
Retained in primary schools with a view to later 
presentation . Ў Я x г Я 342 10:6 
Others: left City, еіс. . . 4 * ? 57 r8 
3,229 


Record Booklets 

The Dundee Post-Primary School Record Booklets were used to 
record the educational history of the pupils concerned in the 
investigation. The original booklets were sent to the Training 
College at the end of each examination period and a duplicate 
record was made of the marks of each pupil. 


Final Follow-up Cards 

Final follow-up cards were devised for the purpose of securing 
a general estimate of the pupil’s progress at the conclusion of the 
first three years of the post-primary course, or when he left school. 
Three types of card were required: for pupils in senior secondary 
schools; for pupils in junior secondary schools; for pupils in 
backward classes. Detailed instructions for completing these 
final follow-up cards were sent to the head masters of the schools 
concerned. A follow-up card is shown in Appendix С. 


CHAPTER II 
7- STANDARDISATION OF THE TESTS 


Tue results of intelligence and scholastic tests may be used either 
in the form of scores or of quotients; that is to say, without or 
with an allowance for differences in the ages of the candidates. 
Аз one of our aims was to find whether the score or the quotient 
should be used in selection, we required a reliable means of trans- 
forming scores into quotients, and this involved the standardisation 
of all the tests. Р 


REASONS FOR хот USING Previous STANDARDISATIONS 


ed The Moray House Tests which were used had already been 

_ Standardised for an English county. At first it was thought that - 

О thee standardisations might be suitable for the purpose of the 
Inquiry, but there were several objections to this procedure. 

| Firstly, there was doubt as to whether the results from a? 
English county with a large rural population would be similar to 
those from a Scottish industrial city. Again, the conditions under 
which tests are given in England differ from those under whic! 
such an experiment is carried out in Scotland. For example, 1t 
is a general rule in England that these tests are given to deter- 
mine which children shall obtain free places in a secondary school, 
and this gives the candidates an incentive to do well. Added to 
this is the fact that the tests have been given for several years in 
the county, with the result that the children have acquired a 
certain familiarity with the form of the questions. 

These difficulties had already been experienced by the Scottish 
Council for Research in Education when it білек ба a national 
survey of a complete age-group in 1932.1 

The comparability of the norms might also be affected by the 
fact that the groups selected for the standardisation of the tests 
in England were at a lower age-level than that of the group tested 
in the Inquiry. b ор 

1 The Intelligence of Scottish Children. 
for Research in Education No. V. Londo 
1933. Рр. 72-79. 


Publications 9f the Scottish Council 
n: University of London Press, Ltd., 
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From a previous survey it was found that the average age of the 
Qualifying pupils in the City was 12 years. As the ages of 73 per 
cent. of the pupils lay within the age-group 11 years 6 months to 
I2 years 5 months it was decided to standardise the tests on this 
group, and to extend the norms on either side of the age-limits. 

Тһе distributions of the children in the age-groups for December 
and June have already been given on p. 3. 


METHOD or STANDARDISATION 


The tests were standardised by a method devised by Professor 
Godfrey Н. Thomson.1 


Comparison or Inquiry AND Ехсілѕн RESULTS 


The previous standardisations enable us to make some com- 
parisons between Inquiry and English children in regard to 
intelligence and scholastic attainment. If Scottish readers find 
the results to be somewhat unexpected, they must remember, in 
addition to the special explanations already given, that the in- 
telligence level of a country is not uniform. 


` 


INTELLIGENCE TESTS 


The comparison on the basis of MHT 20 is given in fig. т, 
which will be intelligible to those who are not familiar with 
Professor "Thomson's technique of standardisation. The lines 
may be regarded as giving the average scores of pupils of different 
ages at various levels of ability. 

The first point to be observed is that the groups on which the 
standardisations are based are at different age-levels, the English 
group ranging from 10 years 9 months ? to 11 years 8 months while 
the Inquiry group range from 11 years 6 months to 12 years 
5 months. "Тһе most outstanding feature of the diagram is the 
superiority of the English group at every level of ability. Another 
feature is the parallelism of the corresponding lines, indicating that 
the age-allowances at the different levels are in close agreement. 
When translated into terms of IQ the difference between the two 


1 G. H. Thomson, “Тһе Standardisation of Group Tests and the Scatter 
of Intelligence Quotients,” British Journal of Educational Psychology, vol. ii, 
Part II, June 1932, pp. 92-137. 

* Only that part of the range which is relevant to the comparison is shown 
in the figure. 

із 


SCORE 


SCORE 


95 PERCENTILE 


5 PERCENTILE 


АСЕ ---- ENGLISH PUPILS 
---- INQUIRY PUPILS 


Fic. r.—Intelligence: Comparison between English and Inquiry 
pupils (MHT 20). 


16 PERCENTILE 


5 PERCENTILE 


AGE 


ENGLISH PUpILS 
т--- INQUIRY PUPILS 


Fic. 2.—English: EI English and Inquiry pupils 
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standardisations amounts to something in the region of 7 to 8 
points. 

At first sight it would appear that the English children are 
superior in intelligence to those of the Inquiry group, but several 
important points must be considered before any such conclusion 
can be drawn. 'These, as already mentioned, include the 
familiarity of the English children with the form of questions 
and the incentive to do well in such tests. 

It is important to note that the process of standardising the 
test on the City age-group makes the mean IQ of this group 100 
and the standard deviation 15. 


ENGLISH TESTS 


Fig. 2 provides a comparison between the results of English 
and Inquiry pupils on one of the English tests. "This is much 
more favourable to the Inquiry group. "They are superior at the 
95-percentile, about equal at the 50-percentile, and show a greater 
rate of increase of score at all levels. 


ARITHMETIC TESTS 
Fig. 3 gives the comparison for MHA 7. Perhaps the most 


striking feature is the small rate of increase in score for increasing 


95 PERCENTILE 


84 PERCENTILE 


5 PERCENTILE 


AGE ENGLISH PUPILS 


---- INQUIRY PUPILS 


Fic. 3.—Arithmetic: Comparison between English and Inquiry pupils 
(МНА 7). 
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аве at the high levels of ability in the Inquiry group. A possible 
explanation is that the clever children reach a uniform standard 
of achievement where variation in score is negligible. This may 
е arise from the teacher's tendency to spend a great deal of time in | 
; bringing the duller children up to a certain level, thus giving little 
= opportunity for the clever child to develop his ability to the full 
2 extent. With the English pupils the rate of increase at the high | 
evel of ability is greater than that at the lower levels, indicating 
_ that the clever children are making greater relative progress than 
е duller children. At all levels the English pupils show 4 
performance superior to that of the Inquiry group. А 
_ the comparisons are all in respect of the December group; which 
is slightly poorer in quality than that presented in June. ln 
. general, however, the results for the June testing are in close 


y 
_ agreement. 


: CHAPTER III 
RELIABILITY OF THE TESTS AND EXAMINATIONS 


CLASSIFICATION ron Srcoxpany Epucation on тне Basis 
or DuPLICATE Quauiryinc EXAMINATIONS 


Ox the basis of the total marks the examiner was asked to divide 
the pupils into four categories: 


A. Those fitted for a senior secondary course leading to the 
Senior Leaving Certificate. Ў 

B. Those fitted for а junior secondary course leading to the 
Junior Leaving Certificate. ` 

C. Those fitted for a junior secondary course of a lower standard 


than B. 
D. Those not fitted for promotion to any of the above types of 
course, d ” 
Category Number of pupils in category Distribution in second 
in first examination examination 
P III 
B 4 
А 126 19 i 
D o 
fa 25 
B 36 
я 82 15 21 
= р o 
в 4 
В 22 
A 157 J^ 1i2 
D 20 
fs о 
B o 
2 135 ТЕ 28 
р 107 
5оо 5оо 


То test the reliability of this classification a duplicate examina- 
Чоп was set to the June group and the scripts of a representative 
13 
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sample of 500 were corrected. The sample was selected by 
arranging the cards of the pupils who sat the first examination in 
order of merit and drawing every fourth. This gave nearly 500, 
the number being completed by drawing a few cards at random. 
The classification of pupils in the two examinations is shown 
on p. 13. 

The agreement, particularly in the A and D categories, is much 
closer than one would expect from some of the data recently 
published on the unreliability of ordinary examinations, and it 


suggests that the correction and classification had been most’ 


carefully performed. 


RELIABILITY COEFFICIENTS OF THE TESTS AND EXAMINATIONS 


Another method of estimating reliability is to find the correla- 
tion between the two series of scores obtained when duplicate 
tests are applied to the same pupils. This is a reliability coefficient, 
and Dr Douglas M. McIntosh determined it for the Qualifying 
examination by using the results for the sample of 500. For the 
intelligence and scholastic tests he found the necessary data 1? 
the fact that nearly 1,400 pupils were common to the age-groups 
tested in December and June. 

If we examine the raw coefficients in the first column of Table I 
perhaps the first point that strikes us is that the correlations for 
the ordinary examinations are unexpectedly high. It is also 
surprising to find that the reliability is higher for English than 
for Arithmetic, particularly in the examinations, where the 
English papers included an essay, 

Change in order of marks from one examination or test (0 
another may be the result of two causes: variation in the standar 
of performance of the individual pupils due to such factors as 
health, fatigue, suitability of the paper; and variation in the 
standard of marking of individual papers by the examiner. In 
the examinations both causes are operative; in the tests only the 
first, for the marking is done by a standardised system. For this 
reason we should expect the reliability of the tests to be higher 
than that of the examinations. 

Reliability coefficients have to be interpreted with some 
statistical caution, since their size depends upon the length of the 
tests. Іп a sense it is unfair to compare the reliability coefficient 
of an intelligence test of 55 minutes with that of an examination 
lasting 34 hours, For a fair comparison we should reduce them 


—— —— a7 —-_ нн ны 
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all to the same length, say an hour; and when this is done? we 
Bet the coefficients in the right-hand column of Table І. The 
tests now rise to the place which one would expect, and Q falls 
from the top of the list to the bottom. Іп this column Q is not 
Ое Qa as in the first column, but an examination with half-hour 
papers in English and Arithmetic. 


TABLE I 


Reliability coefficients (raw and adjusted for length): 
"Tests and examinations 


Test or Reliability Test or Reliability coefficient if all 


examination | coefficient || examination | had been of same length 
oF +933 Е - 4:953 
- +931 А %9% 
Ое 4-920 I +921 
1 +-898 Ое +847 
А 4-877 Qa 4-845 
Qa +864 (Q 4811) 
Qe, Оа, О: N=500 LE,A: N=1,350 


* Ап explanation of the symbols used in the report will be found in the 
Glossary of Symbols in Appendix D. 


STANDARD Errors or MEASUREMENT 


A reliability coefficient depends also on the range of ability in 
the group for which it is calculated, and this factor must now be 
considered. We could correct the above coefficients for range, 
but for various reasons we use a different parameter, the standard 
error of measurement, which is small with a reliable test and large 
with an unreliable one. It has the advantage of being independent 
of the range. We can also calculate what it would be if the tests 
were all of the same length. 


1 The most convincing method would be to use tests and examinations which 
actually are of the same length; but, since this was not practicable, we made 
the reduction by a statistical procedure of which particulars will be found 
in ІР. Guilford, Psychometric Methods. New York: McGraw-Hill Book 


Company, Inc., 1936. P. 419. 
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TABLE II 


Standard errors of measurement: Tests and examinations 


Testor Standard error of Testor Standard error of 
225% measurement * DN. measurement T 
San (percentage) бездер (регсепїаде) 
Е 44I E 3-61 
A 515 A $77 
о 5:30 I 519 
1 5:90 Qe 8-18 
Qe 5:91 Q 8-91 
Qa 8-65 Qa 9:23 
| 


* Comparable for range. 
T Comparable both for range and for length of test. 


The second set of percentages may be taken as giving the fairest 
comparison of the reliability of the various measuring шиштер 
They show that if the standardised tests had an equal chance wit 
the examinations, through equalised range and length, they woul 
be a much more reliable form of measurement. Their relative 
validity is, of course, another matter. A 

In both the tests and the examinations English is more reliable 
than Arithmetic; and the intelligence test is somewhat less 
reliable than the scholastic tests. 

Since there is no variability in the marking of the tests the 
standard error of measurement of 3°61 per cent. in the English 
test appears to be due to other causes, particularly variation 1n 
the pupil’s standard of performance. It is tempting, therefore, 
ignoring statistical rigour, to deduct this from the 8-18 per cent. 
which is the standard error of measurement of Qe; a very rough 
idea of the amount of the unreliability of the English examination 
which is due to variation in the marking is thus secured. ‘This 
would indicate that nearly one-half of the unreliability of an 
ordinary examination is due to variation in the pupils performance. 
If this conclusion were even approximately correct it would have 
alarming implications. In the report of the English International 
Institute Examinations Enquiry Committee! evidence is afforded of 


1 P. J. Hartog and E. C. Rhodes, 


The Marks of Examiners. London: 
Macmillan & Co., Ltd., 1936, of iners 


| 
| 
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the remarkable variability in the marks given by different examiners 
to the same script. То this, it would appear, there has to be 
added an almost equally great variation arising from the fact 
that the single script is only a chance sample of one out of the 
pupils possible performances in similar examinations. 


Те practical significance of a reliability coefficient of +933 
(the raw coefficient for Q) may be expressed in a less technical 
Way as follows: 

Ifa pupil’s true mark in the Qualifying examination were 60 per 
cent., the chance that his actual mark would fall outside the range 


55-65 per cent. is about 1 in 3, 
50-70 per cent. is about I in 22, 
45-75 per cent. is about 1 in 370. 


While we have been at some pains to do justice to the newer 
forms of measuring device, the fact remains that in practice the 
Qualifying examination is much longer than the tests. The first 
Set of percentages of Table II is therefore the one of most practical 
importance; and it shows that, as things are, a carefully set and 
administered examination can bear comparison, in regard to 
reliability, with standardised tests of intelligence and scholastic 
attainment. It should be mentioned, however, that the tests 


which we selected were shorter than some of those now used 
in Scotland, 


CHAPTER IV 


RELATIONS BETWEEN THE MEASURES USED 
AT THE QUALIFYING STAGE 


Tue TESTS AND EXAMINATIONS 


In approaching the problem of selection for secondary education 
it is of interest to know the extent to which the various bests e 
examinations measure the same factors of ability. Clearly, if the 
correlation between two tests is + I, it does not matter which 2. 
we use, and there would be no point in using both. The problem 
is affected by the correlations between the measures in another 
way which the following example illustrates. sch 
Suppose we had four Qualifying tests, A, B, C and D, of whic 

the correlations with success in the secondary school (F) pes 
ТАР = +°7, Tap= 3-6, To= 4-5, трр= 4-4. If an Education 
Committee desired to use two of these tests for selection purposes 
which pair should it select? Taken singly, A and B are the best tests, 
and there would be a temptation to choose them. It is, however, 
quite possible that the best pair to use would be the two worst, 
namely, С and D. This apparent anomaly is due to the fact that 
А and B may be measuring the same aspects of ability, and there- 
fore duplicating each other's work, which would be indicated by 
a high corrélation 74. On the other hand C and D may be 
measuring different aspects of the abilities оп which success 
depends, and this would be shown by a low correlation fop- The 
choice, therefore, depends partly on the correlations of the tests 
with success and partly.on their inter-correlations. 


CORRELATIONS BETWEEN INTELLIGENCE AND THE OTHER TESTS 
AND EXAMINATIONS 


We had two groups, the pupils presented in December 1935 
and those presented in June 1936. We have therefore two 
18 
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estimates in each case, but these were' combined to give single 
correlations! 


TABLE III* 


Correlations with intelligence: Tests and examinations 
(Qualifying and age-groups) 


Correlation between I Qualifying group Age-group 
and (N 22,900) (N 25,600) 
E 4-867 *:897 
Qe 4:846 
A 4:759 4:801 
Qa 4:726 
S 1:880 
Q +855 
Ts +791 
T T5 


* All the differences in correlation within the columns of this table are 
statistically significant. "The method used in testing the significance of differ- 
ences was to transform the correlation coefficients into z and calculate the 
standard error of the difference of z values. ^ 

In the correlation between intelligence and single tests for the Qualifying 
group the smallest ж difference is-074: SE —-026. 1 

In the correlation between intelligence and combinations of English and 
` Arithmetic for the Qualifying group the smallest > difference is 101: SE =.026. 
The х difference for the age-group correlations is 356: SE =-019. 


1 In combining correlations the procedure used was as follows: 
It was first ascertained whether the samples on which the separate 
correlations were determined could be regarded as coming from 


equally correlated populations. 
1+7 


The correlations were then transformed into ж, where х= log, ie 


Тһе weighted average of the ж values was found and re-transformed 
into 7. 
Certain special points in regard to combination of correlations are discussed 
in Chapter VII. 
References: R. А. Fisher, Statistical Methods for Research Workers. Edin- 
burgh: Oliver & Boyd, Ltd., 1936. Р. 207. 
P. R. Rider, An Introduction to Modern Statistical Methods. 
New York: John Wiley & Sons, Inc., 1939. Р. 105. 
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Тһе correlations are relatively high, a fact which is due nl 
to the wide range of ability in the groups, and partly, per sapi; 
to the great care with which the tests and examinations were 
administered. 4 

The results show: 


That intelligence (by which we mean simply whatever d 
measured by the intelligence tests which we used) 18107 
highly correlated with English than with Arithmetic. 1 

That intelligence is more highly correlated with the tests than 
with the examinations, . d 

That intelligence is more highly correlated with the tests an 
examinations than with the teachers’ estimates. 

That the scaling of teachers’ estimates} in such a way d 
they are comparable from school to school, by a metho 


described in Chapter V, increases their correlation with 
intelligence. 


Тһе size of the correlations shows that the intelligence test 
5, to a considerable extent, coveri 


the other tests and examinations; 


mination is better for selection purposes 
than its combination with scholastic tests, 


INTELLIGENCE SCORE AND INTELLIGENCE QUOTIENT 

An important practical 
gence score or the j 
settle this most conclusi 


‘ase of the other coefficients 
It was derived from the inter- 


subjects; and where the accu 


It is based on 2,160 
with the results of direct calc 


тасу of the method could be tested, the agreement 


ulation was very close, 
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same as the ranking on intelligence quotient. "Тһе result is as 
high as +-992. 1 

Тһе fact that this correlation is so high suggests that it would 
matter very little whether we used intelligence score or intelligence 
quotient in selection for secondary education. This does not 
follow: we must have a very high correlation indeed before we 
may draw such a conclusion. 

This important point may be illustrated by actual figures. 
If we have two tests A and B such that rap — 4-992, and if 
лр = 4:700, what can we deduce as to твр? Without making 
assumptions which are normally unsound, it is not possible in such 
à case to determine rpp exactly; all we can do is to fix limits 
between which its value must lie. When we make the calculation 2 
we find that тур must lie between +-604 and 4-784. Even if 
Tan Were as high as 4-:999, all we could say is that 7вр must lie 
between +-668 and +731. : 

It does not follow, of course, that the correlations will differ 
аз much as the above limits permit, and some results for the 
Qualifying group giving actual variations are shown on p. 22? 


(№ 22,900). 


1 This correlation was not calculated directly, but derived by means of the 
following formula from the correlation of intelligence score with age: 


Тпот = 
/4 Titi. ба 

— = 27а 5 #18 

N M? "MM, M? 


Мү=тгеап of intelligence scores; o1=standard deviation of intelligence scores; 
a=age, 
й йм 
This formula assumes that, in expansions, powers of a higher than the 
second may be neglected, d; representing the deviation of an intelligence score 
from the mean. 
Reference: G. Udny Yule and M. С. Kendall, An Introduction to the Theory 
of Statistics. London: Charles Griffin & Co., Ltd., 1937 
Chapter xvi. 
2 The limits for rpp are rapran + УІ E bris 
Reference: С. Udny Yule and М. С. Kendall, An Introduction to tbe Theory 
of Statistics, p. 280. - қ 
з The correlations with IQ were obtained by the method described іп 
the footnote on p. 20. we 
The differences between the correlations for IQ and I are barely significant 
statistically. Тһе = differences аге -056, -o5o and -o61: SE =.028. 


22 SELECTION FOR SECONDARY EDUCATION 


Correlation with IQ I 
S 1:892 4-880 
Q +841 +855 
Ts 7:767 +791 
Average 4-833 4-842 


The conclusion that it makes little difference whether we use 
intelligence score or intelligence quotient will be confirmed later 
by the follow-up correlations. 


CORRELATIONS BETWEEN TESTS AND EXAMINATIONS 
TABLE IV 


Correlations between tests and examinations 
(Qualifying group: N —2,900) 


Correlation between т 
Е апа Ое +:866 
А апа Оа 4:810 
S and Q 4-:898 , 


The correlations are again high, but even that of nearly 79 
between S and Q leaves ample room for differences in the selective 
value of the two types of measure. One interesting point that 
emerges is that the correlation between Е and I is almost exactly 
the same as that between E and Qe. 


CORRELATIONS BETWEEN ENGLISH AND ARITHMETIC 
TABLE V 


Correlations between English and Arithmetic 
(Qualifying and age-groups) 


Correlation чаш Age-group 
between c prt m (ЇЧ = 5,600) 
Qe and A %-737 
Qe and Qa +721 
Е аад +708 +:766 
Е апа Оа +664 
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Тһе higher correlation for the age-group is due to the wider 
range of ability. . 
` The correlation between ability in English and ability in 
Arithmetic is a little above -7; but it is interesting to notice 
how much its size depends on the nature of the measures 
used. Even if we take the highest correlation of +766, it is clear 
that a knowledge of a pupil’s standing in English gives very little 
guidance as to his standing in Arithmetic. From the graph given 
in fig. 27! it can be seen that the error of prediction would be 
about 64 per cent. of what it would be by pure guess-work. 


CORRELATIONS WITH TEACHERS’ ESTIMATES 
TABLE VI* 


Correlations with teachers’ estimates 


(Qualifying group: N = 2,900) 


Correlation between T 
Ts and О +906 
Ts and 5 +-866 
T and Q +°774 


* The first two correlations were obtained by the method described in 
the footnote on p. 20. 

The differences are all significant statistically. The ж difference between 
the first two 18.188: SE =-030. 


The correlation of nearly -91 between Ts and Q is gratifying, 
but it gives no ground for the belief that the uniform examination 
could be replaced by the teachers’ scaled estimates. Apart from 
the difficulty that the scaling is done on the basis of the examina- 
tion marks, the correlation is still far too low. 

Again we see the very significant rise in correlation that results 
from the proper scaling of the teachers’ marks; and, as is to be 
expected, the scaled estimates correlate more highly with the 
examination than with the scholastic tests. 


З.Р; 1247 
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EFFECT OF AGE ON THE CORRELATIONS 


Correlations of the kind with which we are dealing may be 
considerably affected by lack of homogeneity of the groups. The 
Qualifying group is homogeneous in regard to the school class of 
the pupils, but very heterogeneous in regard to age. It is there- 
fore necessary to ascertain whether the correlations are inflated 
by the fact that the data correlated have a common correlation 
with age. е 

One way in which we can do this is by comparing the correla- 
tions for the complete Qualifying group with those for a group 
which is reasonably homogeneous in regard to age; and for this 
purpose we chose the Qualifying pupils who fall within the age- 
group II years 6 months to 12 years 6 months. The correlations 


for the Qualifying group thus analysed into its component parts 
are given in Table VII, 


TABLE VII * 
Analysis of correlations for Qualifying group 
D es 
Complete group Middle ere т 6 and 

pa 10.11-14.9 11.6-12.6 over 1216 

(N-290) | (N=2,170) | (М-739) 
I and E +-867 4-928 4-878 
I and A +-+759 4-759 4:773 
I and Qe 43846 4-842 +843 
I and Qa 3-726 3:727 +719 
Tand Q +855 +853 +856 
E andA 4:708 +:718 4-:678 
E and Qe 4:866 4:905 4-876 
E and Qa 4:664. +-671 4:640 
А and Qa 4-810 4-840 +835 
А апа Ос +2737 4-743 +714 
Ое апа Оа 3721 4-726 +-701 
T andI 4715 477 4-703 
T andQ %:774 ++780 +756 
Average +773 +-785 4-767 


* 'The difference in average correlation between th 
extremes is not significant statistically. Тһе oth 
significant. The ж difference between the complet. 
+031: SE=-008. Between the middle and the extre; 


е complete group and the 
er average differences are 
€ group and the middle is 
mes it 18-045: SE =.012. 


„м 
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"The Table brings out the interesting result that in a Qualifying 
group the correlation is highest in the middle, and is considerably 
lower in the extremes; that is to say, it is highest where the age- 
scatter is least. The difference is no doubt due to the fact that 
in Qualifying classes the old children are usually dull and the 
young children clever. It is clear, too, that the correlations fora 
Qualifying group are much the same as those for a group which is 
homogeneous in regard both to age and to school class. 

The above results can be shown from a different angle by using 
the correlations with age. We should expect a negative correla- 
tion in the extremes, but, in fact, all the correlations with age for 
the Qualifying group are negative. A small selection, calculated 
by Dr McIntosh, is given below: 


Correlation 1 between Middle Extremes 
age and % (N 22,217) (№ 7737) 

Qe --086 —:129 

Qa —-112 —:129 

о — +102 —-156 

Average —-100 -:138 


We may conclude from this evidence that even in that part of 
2 Qualifying group which is fairly homogeneous in regard to age 
the correlation with age is negative. "This means that the older 
Pupils are, on the whole, slightly poorer in ability and attainment 
than the younger. 


USE OF PARTIAL CORRELATION METHODS 


By the method of partial correlation we can find what the 
correlations would be if all the pupils were of the same age; and, 


* All the correlations here are significantly different from zero. The prob- 
ability that they could be obtained by random sampling from a population of 
zero correlation is of the order P —-0003. 

‚ In the few cases where it was necessary to test the significance of our correla- 
tons the formula used was 


rVn-2 


1-73 

The differences between the columns are not significant statistically. The 
Z difference for the average is -039: SE =-024. 

References: R. A. Fisher, Statistical Methods for Research Workers, pp. 195 


and 205. 
Р. R. Rider, An Introduction to Modern Statistical Methods, 


pp- 83 and 93. , 
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as would be anticipated from the above results, the differences 
from the raw coefficients are extremely small. Out of five cases 
calculated by Dr McIntosh the largest difference was “002. 


EFFECT OF DIFFERENCES IN SCHOOL CLASS ON THE 
CORRELATIONS FOR THE AGE-GROUP 


The Qualifying group is homogeneous in regard to school class 
but heterogeneous in age; the age-group, on the other hand, is 
relatively homogeneous in regard to age, but heterogeneous in 
regard to school class! Our data provide us with an opportunity 
for making an analysis of the correlations for such a group; and, 
while this is not of much importance for our selection problem, 
it is of considerable interest in connection with the interpretation 
of correlation coefficients. In this case we shall count the pupils 
in the Qualifying class as the middle, and those in the classes above 
and below as the extremes. 


TABLE УШ 


Analysis of correlations for Qualifying and age-groups combined 


m 


Average of 
Group N three 


+ * 
correlations 


/ 


Age-group outside Qualifying classes. .| 344 +844 
Complete age-group . $ П > al 5,608 T3821 
Combined age-group and Qualifying group .| 6,335 +817 
Qualifying group in age-group s .| 2,194 +:802 
Complete Qualifying group 5 5 „| 2,921 47778 
Qualifying group outside age-group . " 727 4776 


" 
TIE, 714 and rg, 


" While in the Qualifying Broup the correlation in the extremes 

is lower than that in the middle, or relatively homogeneous part, 

the opposite holds for the age-group.* "The position is, therefore, 
1 Cf. pp. 2 and 3. 


? The difference is statistically. significant. The ж difference is -131: 
SE =.016. 
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that at the Qualifying stage in Scotland the inclusion of the old 
and young pupils in a group which is homogeneous in regard to 
School class lowers the correlations; but that the inclusion of 
pupils from higher and lower classes in a group which is homo- 
geneous in regard to age raises the correlations. In this connection 
it should be mentioned that in the case of the age-group the 
standard deviation in the extremes is higher than the standard 
deviation in the middle, the opposite being true of the Qualifying 
group. 

, From the Table we see that the correlation for an age-grouping 
is higher than that for a class-grouping, the difference in correla- 
tion being about +04.1 


Tue Data FROM THE Ratinc CARDS 


INDUSTRY 


(In collaboration with Miss J. I. W. Robertson, M.A.) 


The Qualifying teachers were asked to grade all pupils in regard 
to industry, using a five-point scale in which the significance of 
the letters was as follows: 


А Exceptionally high. 

B High—decidedly above the average. 
C Average. 

D Low—decidedly below the average. 
E Exceptionally low. 


Тһе leaflet of instructions gave the percentages of pupils who 
would fall into the various categories in а normal class. 

Using the June group, Miss Robertson calculated correlations 
between these industry gradings and intelligence quotient, Qualify- 
ing examination marks, and health. She dealt with boys and girls 
separately. 

All correlations which we have calculated for data on five-point 
scales have been corrected for broad grouping.? 


1 The difference is statistically significant. The æ difference is -119:. 
SE =-013. 
2 References: К. Pearson, “Оп the Measurement of the Influence of Broad 
Categories upon Correlation,” Biometrika, vol. ix, 1913, 
р. 116. ү b 
K. J. Holzinger, Statistical Methods for Students in Education. 
New York: Ginn & Co., 1928. Chapter xiv. 
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TABLE IX * 


Industry: Correlations with IO, O, H 
(June Qualifying group) 


Correlation Boys Girls All 
between 
industry 

"and N T N r N 


622 
0 953 | 4-598 | 987 | 4647 | 1,940] + 

Q 931 | +568 | 956 | +664 | 1,887 | +616 
H 736 +112 763 +140 | 1,499 | +42 


* Тһе correlation with health is significant statistically, P being far pus 
‘or. As far as the differences between boys and girls are concerned, a 
difference between the correlations with Q is significant; ж difference E 
*1$5: SE =.045. In the other two cases the differences are not significant, 
though that for IQ is nearly so; 2 difference is -o80: SE =:045. 


It may be that there is a tendency for teachers to ascribe a xm 
degree of industry to a child who is doing well in class, while his 
Success may be due rather to superior ability or home encourage- 
ment. On the whole, however, we felt that the industry gading 
were much more reliable than those for interest, which we shal 

= discuss presently. 

Taking them at their face value, they show a correlation of about 
:6 both with intelligence and scholastic attainment and only a 
small positive correlation with health. From our point of view 
the results offer hope that industry may be of some value in fore- 
casting success, since its correlation with the other measures is 
such as to suggest that it may cover a part of the field which they 
leave untouched. Perhaps the result that gives most food for 


given in Table X. 


The Table gives the percentagesfalling into the various categories. 
They are comparable as between industry and Qualifying examina- 
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tion marks, for the latter distribution was cut in such a way that 
the dividing lines between the categories are at the same o distance 
from the mean in both cases. "Гһе means at the foot of the 
columns are found by giving five points for an A grading, four 
points for a B and so on, and may be taken as giving a rough 
comparative measure of the quality of the groups. 


TABLE X 


Industry: Comparison of boys and girls 
(June Qualifying group) 


Boys Girls 
(N=931) (N=956) 
Category 
Ind. Q Ind. Q 
A 3:0 2:0 30 2:9 
B 26:5 30:0 26-8 30:5 
C 53:6 48-6 568 504 
D 15-0 169 15 14-8 
E r8 2:6 1:9 r2 
Mean 314 3-12 318. 319 


The differences between boys and girls are statistically significant | 
both in industry and in Qualifying examination marks, the former 
difference being nearly four times its standard error. On the 
whole, therefore, the results support the view that girls are more 
industrious than boys, though the difference is not very great. 
А closer study of the table shows where the difference mainly lies. 
"The good boys are nearly as industrious as the good girls, but the 
boys of poorer ability are less industrious than girls of the same 
type. 


PUPILS' INTERESTS 


(In collaboration with Mr G. A. Crabb, B.Sc.) 


Predictive Value of Teacher? Interest Gradings 
With a view to exploring the value, in guidance and selection, 
of information as to the direction of the pupils’ interests, we asked 
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the primary teachers to grade each Qualifying pupil on a five-point 
scale in regard to interest and attainment in the three main groups 
of school subjects—humanistic, realistic and practical. The 
meanings of the letters were the same as in industry. Д 

Мг Crabb made a study of these interest gradings; and, while 
his results did not encourage us to proceed with the statistical 
working-up of this part of our data, he continued his work on the 
subject of children’s interests until his entry on war service. The 
following is an account of the part of his investigations which bears 
upon the present Inquiry: his further results he will, no doubt, 
publish independently. 

The group chosen was that presented in June, which showed a 
good distribution of pupils among the five categories. When, 
however, the gradings were compared with those for attainment 
there was ground for suspicion that the agreement was greater 
than was probable on the basis of a true assessment of interest. 
Mr Crabb made the comparison in two ways: by finding the 
numbers of pupils who came into the same category both for 
interest and for attainment, and by finding the correlation, 
corrected for broad grouping, between the interest and attainment 
gradings. By the first method he found that no less than 84 per 
cent. had the same grading in both. The correlation between 
the interest and attainment gradings in humanistic subjects was 
+94. " 

Mr Crabb planned an interesting subsidiary experiment to fin 
how far the teachers estimates of interest corresponded with 
the children's own ideas on the subject. A new group was formed, 
consisting of the Qualifying pupils in five representative schools. 
These pupils were graded by the teachers in exactly the same way 
as the Inquiry group had been, They were then given a simple 
Interests questionnaire. 

Sixteen school subjects or parts of subjects (for example, Poetry; 
Composition, Spelling) were listed, and the pupil was asked to! 
indicate by underlining whether he liked the subject, disliked it, 
or was uncertain whether he liked or disliked it. For purposes of 
scoring, a like was credited with three marks, a not sure by two 
marks, and a dislike by one mark. The subject selected as favourite 
was given an extra mark. м 

Тһе school subjects were then divided into humanistic, realistic 
and practical, so that the pupils score in each group could be 
obtained. Ву cutting the distribution of scores in such a way that 
the border-lines were at the same c distance from the mean both 
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in the teachers’ gradings and in the pupils’ own scores, a comparable 
five-point grading, made by the pupils themselves, was secured. 

TABLE XI 


Interest: Comparison of gradings made by teachers with those made 
by pupils (special experimental group: N —227) 


Correlation between | Percentages of pupils 


Group of subjects teachers’ and pupils’ | іп same category in 
gradings both gradings 
Humanistic . я 4:26 34:8 
Realistic . А . +46 44:0 
Practical . " Е +13 39:6 


The average correlation between the teachers’ gradings in 
interest and attainment was +95, which is very close to that 
obtained for the Inquiry group. The average correlation between 
the pupils’ gradings of their own interests and the teachers’ grad- 
ings for attainment was +:40. 

These results show that there is great disparity between the 
teachers’ estimates of children’s interests and the children’s own 
ideas; and they tend to confirm the suspicion that the teachers’ 
gradings were largely influenced by the pupils’ attainments. ‘That 
such may be the case is no reflection upon the care with which the 
teachers made their estimates: the fault lay in our technique, 
which set them an impossible task. We decided, however, that 
instead of calculating follow-up correlations for the interest data 
on the rating cards, Mr Crabb should follow up the pupils in his 
experimental group with a view to comparing the prognostic 
value of the teachers’ and pupils’ gradings. This project had to 
be abandoned on account of his entry upon war service. 


Pupil? Order of Preference of School Subjects 

‘The method of scoring used by Mr Crabb enabled him to deter- 
mine comparative interest scores for the various school subjects, 
and, though the number of pupils was rather small, fig. 4 is 
presented as a contribution to the mounting evidence on this 
topic. It gives the order of preference for a group of pupils at 
the Qualifying stage in a Scottish industrial City, and shows also 
the range of variation in the placing of the subjects by individual 
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pupils. The centre points give the mean placing of the subjects, 
and the horizontal lines cover a range of +30. 


PAINTING 
PHYS. EDUCN. 
DRAWING 
HANDWORK 
ARITHMETIC 
READING 
HISTORY 
POETRY 
WRITING 


GEOGRAPHY 
SPELLING 
NATURE STUDY 
COMPOSITION 
MENTAL ARITH. 
INTERPRETATION 
GRAMMAR 


Fic. 4.—Pupils’ preferences in school subjects. 


The vertical line between 170 and 210 divides subjects where the 
number of /ikes is greater than the number of dislikes from those 
where the reverse із the сазе. It could hardly be assumed that this 
gives a border-line between absolute liking and dislike; one suspects 
that the child gives his judgment with the reservation ‘as far 25 
school subjects go,’ Still, the results confirm the view | that 
Grammar is a distasteful subject to the majority of pupils at 
this age. Yet even here, as a cautious boy put it, ‘everything 
depends on the teacher *; in one class the gradings for Grammar 
were: like, бо per cent., not sure, 40 per cent., dislike, o per cent» 
favourite subject, 30 per cent. 

Comparison of Interests of Boys and Girls , ? 

The number of pupils in this experiment was too small to yield 
reliable results, but, generally Speaking, they give no ground for 
the belief that there is any significant difference between the 
educational interests of boys and girls at this stage. There were 
small differences in the order of preference of school subjects but 
only one or two stood the test of statistical significance. Nor, as 
is shown in Table XII, is there any statistically significant difference 
between boys and girls in the three main groups of subjects. 


Pr pd 
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TABLE XII 


Interest: Comparison of mean scores of boys and girls in humanistic, 
realistic and practical subjects 
(special experimental group: N —227) 


Boys Girls 


Group of subjects (N=116) (Nu 


Humanistic . Д x 2:31 2:39 
Realistic ў š . 246 242 
Practical Я " а 2:75 2:83 


Аз might be expected, both groups show the greatest preference 
for the practical subjects, and a slightly greater preference for 
the realistic than for the humanistic subjects. 


СНАРТЕК У 


TEACHERS ESTIMATES 
(In collaboration with Mr John T. Bain, M.A., B.Sc.) 


VARIATION IN STANDARD FROM $споог то SCHOOL 


ee : have 
Ox a preliminary survey the teachers’ estimates appeared to } 
the following characteristics: 


They were generally too high. ! 

They tended to over-estimate the attainments of the weak 
pupils and under-estimate those of the good pupils. | 

Тһе standard varied considerably from school to school. 


int of 
The first of these features is not very serious from the point ie 
view of follow-up correlations. The second decreases the es E. 
. given to teachers’ estimates when they are combined with " the 
measures having a larger standard deviation. The third “bl A 
most important, for it would lower the correlations considera” Te 
To ascertain the extent of the variation in standard, Mr hools 
analysed the data for a representative group of twelve $C 
comprising about 850 pupils in all. sin 
At the moment we are concerned only with the column Qa. 
Tables XIII and XIV which are headed Те, Qe, Ta and тА 
They show that in English the teachers estimates аге, ОП FO 
average, 15 per cent. higher than the scores made by the 54 a 
pupils in the uniform examination; in Arithmetic they are 25 P 
cent. higher. - E 
In English the teachers? estimates are much less widely scatter 
than the examination marks, the average standard deviations beiDE 


B à 5 | ег 
12-3 and 18-4 respectively; in Arithmetic the difference in scatt 
is very much less. 


‘The variations in standard from school to school are very 
striking. In English the teachers estimates for School 4 are oD 
the average 1:9 per cent. above the mean mark scored in the 
examination; in School 9 they are 30:2 per cent. higher. The 
difference in standard of marking between these two schools 15 


1 Cf. Chapter VII. 
» Р 34 
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therefore more than 28 per cent. In Arithmetic the maximum 
difference in standard is 22 per cent. 

TABLE XIII * 


Teachers’ estimates in English: Comparison with marks in English 
scored by the same pupils in the Qualifying examination 


Te Qe Tes 

School Tano 

M c M с M с 
: 667 | 147 | 566 | 199 | 562 | 189 | +-94 
2 604 | 123 | 421 | 182 | 378 | 155 | +72 
3 649 | 107 | 620 | 172 | 60-2 | 158 | 4-79 
4 639 | тоо | 62-0 | 163 | 60:7 | 161 | 4-89 
5 675 | iri | 60-7 | 167 | 578 | 169 | 4-87 
6 65:3 | 110 | 522 | 184 | 52-3 | 171 | +82 
7 747 | 168 | 581 | 189 | 542 | 184 | 4-85 
8 652 | 15:5 | 458 | 221 | 472 | 211 | ді 
9 761 | 13:2 | 459 | 165 | 458 | 181 | 482 
10 бг1 | 18-3 | 383 | 19:5 | 371 | 176 | 4-83 
n 605 | 79 | 395 | 154 | 364 | 12-4 | 4-60 
12 672 | 96 | 44:3 | 181 | 44-7 | 15-1 | 4-76 
Average . -| 66-1 | 12-3 | 51:1 | 184 | 496 | 171 | 482 
Slumped schools | 66-1 | 13-6 | гі | 203 496 | 191 | 477 


* The numbers of pupils have been omitted to avoid the possibility of the 
identification of particular schools. "The averages at the foot of the columns 
are weighted averages. Marks are in percentages. 


It is thus confirmed that teachers’ estimates have the three 
features mentioned above. In regard to the first it has to be kept 
in mind that if the estimates are higher than the examination 
marks it does not follow that it is the teachers’ standard that is 
wrong, while the difference in scatter may be partly due to the 
practice of basing estimates on the average of a large number of 
class tests. Such averages will normally have a smaller scatter “ 
than the marks in a single examination.1 


i$ 2 E 
Since o sag, )7 Rog +2700; 
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TABLE XIV 


Teachers? estimates іп Arithmetic: Comparison with marks in Arithmetic 
scored by the same pupils in the Qualifying examination 


Ta Qa Tas 
School : TQaTa 


M c M с M с 


боо | 24-3 | 524 | 209 | 528 | 220 | +88 
69:5 | 165 | 359 | 160 | 37:0 | 172 +75 
713 | 34 | 516 | 173 | sr2 | 176 | +88 
728 | 165 | 585 | 163 | 576 | 158 +78 
720 | 194 | 50-4 | 181 | 476 16-6 
65 | зт | 555 | 15:7 | 537 | 149 | #7 
791 | 138 | 517 | 187 | 496 | 150 | +75 
745 | 20-3 | 40:3 | 172 | 426 | 208 42% 
774 | 229 | 425 | 147 | 425 |164 | +7 

62:6 | 22:8 | 370 | 220 | 388 | 248 782 
58-6 | 10-7 | 266 | 13:3 | 230 | 123 +74 
65-8 | 18-1 | 381 | 19:9 | 40-1 21:6 +6 


моя ы 
M HOO ON Ond зын 


Average . . | 704 | 176 | 456 | 179 | 454 | 183 +77 
Slumped schools | 70-4 | 18-8 | 456 | 199 | 454 | 2071 +73 
[E 


Comparison or VARIATIONS IN STANDARD For ÉNGLISH 
AND ARITHMETIC 


'The question is whether a school that marks leniently in English 
marks leniently to the same extent in Arithmetic, and whether 
a school whose marks are widely scattered in English shows 4 
similar wide scatter in Arithmetic. These are, as we shall 566; 
important points in connection with the scaling of teachers 
estimates. 

In making the comparison it is difficult to obtain figures which 
can be readily compared in view of the fact that the teachers 
marks in Arithmetic are on the average 24-8 per cent. higher than 
those in the examination, while in English the difference is only 
15 per cent. We have therefore added 9:8 per cent. to the Qa 
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average, and made a corresponding correction in the standard 
deviations.! 
TABLE XV 


"Teachers? estimates in English and Arithmetic: Comparison of 
variations in standard of marking 


M с 
School 2 

Te - Qe Ta- Qa Te - Qe Ta - Qa 
I + 8-1 + 68 + 6 +34 
2 +18-3 %23:8 - 1 + 55 
3 + 29 976 =E “29 
4 + 19 + 45 -5 + 2 
5 + 6:8 +11:8 +2 +13 
6 +131 + 32 = 1:6 = 46 
7 +16:6 4176 -23 -49 
8 +194 +244 - 8 +31 
9 4302 ` +251 +275 482 
10 +228 4158 +46 + 8 
II +210 +222 -17. -2:6 
12 +229 +179 -27 -r8 


The answer to our question is, therefore, that we should make 
serious errors if we assumed that a school which marks leniently 
in English marks equally leniently in Arithmetic, or that a school 
whose marks are widely scattered in English shows a correspond- 
ingly wide scatter in Arithmetic. 


CORRESPONDENCE BETWEEN ORDERS ОЕ MERIT IN TEACHERS’ 
ESTIMATES AND Unirorm EXAMINATION 


Mr Bain calculated correlations between teachers’ estimates and 
Qualifying examination marks for each school, for English and 
Arithmetic separately. These are given in the last columns of 
"Tables XIII and XIV, from which it will be seen that the average 


1 It should be noted that the standard deviations for the slumped schools in 
Tables XIII and XIV have a different meaning from the average standard 
deviations. For this correction we use the average standard deviations. 
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correlations for English and Arithmetic are 4-82 and 7 
respectively. "The difference is statistically significant, so we may 
conclude that there is a closer correspondence in English than in 
Arithmetic Іп neither subject, however, is the correspondence 


аз close as we might expect in marks which purport to measure the 
same abilities. 


MzrHüops or Repucine TEACHERS? ESTIMATES FROM DIFFERENT 
SCHOOLS ro THE SAME STANDARD 


The above results show that teachers’ estimates, as they stand, 
should not be used in selection and guidance. Before using не” 
Some means must be found for making them comparable in standard, 


and there are two ways in which this improvement might be 
effected. 


GUIDANCE TO TEACHERS AS TO STANDARD FOR ESTIMATES 


Тһе first line of attack is to devise ways in which teachers could 
be helped to use a uniform objective standard of marking, but all 
that we could attempt in this direction was a preliminary investiga- 
tion of one of the possible methods.? We asked the teachers to 
give estimates on a five-point scale where each category had a 
definite significance, the percentages of a normal class that would 
fall into each category being indicated in the leaflet of instructions. 
"The gradings were made for attainment only, taking no account of 
the pupils’ ages.3 

Mr Bain investigated the question whether these letter estimates 
were on a more uniform standard than the percentage marks. He 
took the five-point gradings in attainment in humanistic and 
realistic subjects, determined the points on a percentage scale 
which corresponded to the means of various categories, and thus 
found a rough measure of the means and standard deviations for 
the various schools. ‘These are shown in Table XVI. 

We cannot attach much importance to the fact that the averages 
of the means on the five-point scale are much closer to the corre- 


1 z difference is -136: SE =:049. 

2 A useful discussion on this subject will be found in the report of the 
Swedish Committee of the International Institute Examinations Inquiry— 
Г. Wigforss, Tbe Entrance Examination in view of Later School Performances, 
Stockholm: P. A. Norstedt & Söners Forlag, 1937. P о. 

3 For the significance of the letters see p. 27. 
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% TABLE XVI 


Teachers’ estimates on five-point scale: Improvement in 
uniformity of standard 


English - Arithmetic 
School 
choo! " Te ы та 
„у |(5-point} Qe _ | G-point| Qa 
(margs) scale) (Шан) scale) 


бру | 55:2 | 566 | 690 | 586 | 524 
боф | 458 | әзі 69:5 498 35:9 
64:9 442 62-0 7r3 514 51:6 
63:9 54:8 62-0 72:8 60:0 58-5 
675 531 бол | 720 | 526 | 504 
65:3 517 52:2 68:5 52:7 555 
7 | қа | Ser | Tot | 558 | SET 
652 | 509 | 458 | 745 | 539 | 493 
761 | 504 | 459 | 774 | 509 | 425 
61-1 46:6 38-3 62-6 442 370 
боб | 465 395 | 586 | 450 | 266 
67:2 48:2 4473 65.8 50:9 38-1 


нны 
ом ом ON OV d о оь н 


Average М | 66-1 49:6 51-1 704 51:7 45:6 
Average с 123 1473 203 176 | 154 199 


sponding figures for the uniform examination, for this depends on 
the statistical adjustment. To find the improvement in standard 
of marking, both forms of estimate were reduced to the same means 
as the corresponding examination marks and the average differences 
calculated. These are given in Table XVIL! from which it will 
be seen that the improvement by the use of the five-point scale 
is 1-1 per cent. That this is not sufficient is confirmed by a 
scrutiny of the results for the individual schools. In English, with 
the five-point scale, School 3 under-estimates by 17:8 per cent., 
while School 10 over-estimates by 8:3 per cent. 

Mr James C. Kidd, M.A., B.A., investigated the correlations 
of the five-point estimates with later success in the senior secondary 


1 p.43. 
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courses, and found that they were all lower than the corresponding 
correlations for the percentage estimates. p 

We conclude that a considerable improvement in ted 
would be necessary before the five-point method would yie 
results that could profitably be used in practice. 


THE SCALING OF TEACHERS’ MARKS 


The second method is to take the teachers’ estimates ac 
are and to reduce them to comparability by statistical proce n 
For this purpose it is not sufficient to use standard scores, sin а 
this would assume that the pupils of all the schools had the = 
mean level of attainment. Clearly we must transform the man бы; 
marks in such а way that the mean and standard deviation o i 
transformed marks for any school will be the same as the pes ol 
standard deviation of the marks made by the pupils of that scho a 
in a uniform test or examination in the same subject. This са 
be done by the transformation 1 


Ts- Ma (T - Mj) 
OT 


and this, in our view, is the soundest method to adopt. It ei 
moreover, simple enough in practice. Since the equation is liner 
the constants are easily calculated and a transformation Table ca 
be quickly made. he 
It has, unfortunately, one disadvantage: in certain cases 1 
transformed marks fall outside the limits of the percentage sca er 
Тһе statistician would readily reconcile himself to this, but it 1 
doubtful whether an Education Committee would view with pni 
any method which ascribes а negative mark to a pupil. Suc 
negative marks are, of Course, not so absurd as they seem at first 
sight. They do not mean that the pupil has less than а 2610 
knowledge of the Subject, merely that he is a certain distance 
below some arbitrarily chosen point of reference. Apart from 
the possible misunderstandings, however, there is a practical 


difficulty in the use of negative scores in that they are a fruitful 
source of error. 


METHOD USED ім THE Inquiry 

We experimented with variou 
above difficulty, keeping in mind 
1 К. J. Holzinger, Statistical Methods 


5 methods of overcoming the 
the fact that the system used 
Sor Students in Education, р. 121. 
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should be one that could be carried out quickly by the ordinary 
Clerical staff of an Education Office; and in the end the following 
procedure was adopted. We took the teachers’ estimates and the 
examination marks for a school and equated the quartiles and ends 
of the ranges, giving a graph of the type shown in fip. s. A 
Separate graph must of course be made for each school. ІҒ the 
raw mark falls at P we read the transformed mark at Q. 


TEACHER'S RAW MARKS 


TEACHER'S SCALED MARKS 


Fic. 5.—Graph for scaling teachers’ marks. 


While the equation of the quartiles is not open to serious 
Statistical objection, it was with some hesitation that we decided 
to use the highest and lowest scores. We must have two fixed 
points outside the quartile range Q,-Q, if we wish to avoid 
negative scores, yet in using the ends of the ranges we felt that we 
were too much dependent upon accident: a pupil who had made 
an exceptionally low score in the examination would pull down 
the marks of all pupils below the lower quartile. "Тһе procedure 
was, however, given a very searching test through its application 
to the whole Qualifying group, and our justification for offering 
it as a practical solution of the difficulty is that with no pupil 
did it appear to operate unfairly. 


2* 
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DIFFICULTIES IN THE Use oF SCALING SYSTEMS 


The main practical difficulty is that ло scaling system can work 


satisfactorily with small numbers, and these may occur in two 
cases: 


In country districts the number of Qualifying pupils in any 
one school may be very small. 

In urban systems where the three-track plan is used the 
numbers in the lowest section may be small, even in a 
school of considerable size. 


Of these we had to contend only with the second, and the 
difficulty did not prove to be serious. We ascertained what pro- 
cedure had been used in making the estimates; and fortunately 
most schools had given the marks for the different sections on the 
same standard. In a small number of schools the C section pupils 
were marked more leniently, and the estimates had to be scaled 
separately. 

The difficulty arising from the three-track plan can be overcome 
by requiring the teachers to give their estimates on the same 
standard for all pupils. For the small rural school the system О 
scaling is quite impracticable; and, unless uniformity of standard 
can be secured in other ways, the use of teachers’ estimates should 
be abandoned. 

A third possibility of breakdown in the scaling system would 
Dot arise if our final recommendations are accepted. Іп certain 
Areas a special examination is held for admission to secondary 
schools or for the award of bursaries, and it may happen that the 
number of candidates from a particular school is very small. We 
believe, however, that any sound system of guidance and selection 
at the Qualifying stage demands the examination of the complete 


ү шу group, by which we mean all the pupils in the Qualifying 
classes. 


IMPROVEMENT EFFECTED BY SCALING 


The improvement 


effected b i inferred from 
"DEI QUIT and XLy ected by scaling can be inferre 


4 › where the school means of the teachers’ 
transformed estimates are given. "Тһе standards for the schools 
are now closely in conformity with the marks scored by their 
pupils in the uniform examination, and the scatter is approxi- 
mately that given by the examination. If we make all the average 
means the same, we obtain a fair comparison of the average 
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difference between the school means and the examination means 
for the three types of estimates with which we have experimented. 


TABLE XVII 


"Teachers! estimates: Average difference between school means and means 
of Qualifying examination marks for various types of estimate 


Average difference (per cent.) 


"Type of estimate 


English Arithmetic 
Unscaled percentages . 74 67 
Five-point scale . 6:3 56 
Scaled percentages 155 1:6 


, Perhaps the most important result of the scaling is that the large 
differences in standard have entirely disappeared. 


SEPARATE SCALING FOR ENGLISH AND ARITHMETIC 


So far we have examined the procedure where English was 
scaled on English and Arithmetic on Arithmetic. This involves 
a good deal of labour, which could be halved if we simply scaled 
the totals for the two subjects. To consider whether this would 
be legitimate or desirable we have to introduce a somewhat 
cumbrous notation: 


Let Te =teachers’ raw estimates in English, 
Tes=teachers’ estimates in English scaled on English, 
and similarly for Ta and Tas, i 
T -teachers raw estimates in English + Arithmetic. 
Т is therefore equal to Te + Та. 


We can arrive at the teachers’ scaled estimates for English + 
Arithmetic in two ways. Te may be scaled on Qe, giving Tes, 
and Ta on Qa, giving Tas. This method gives Ts = Теѕ + Tas. 
We may, оп the other hand, scale Te 4- Ta on Qe +Qa, obtaining 
a result which we shall denote by Ts. 

Our problem is to ascertain whether, for purposes of guidance 
and selection at the Qualifying stage, it would be legitimate or 
desirable to use T$ instead of Ts. This is a question to which 
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the most satisfactory answer would have been obtained by using 
both methods of scaling and by calculating follow-up correlations 
for the resulting scaled estimates. This was not done, partly 
because of the labour involved, and partly because we have other 
reasons, which we shall state presently, for preferring Ts to Ts. 
Without actually scaling we can roughly deduce the predictive 
value of Ts as compared with Ts. The method used has a some- 
what mathematical appearance; and as it is too long for a foot- 


note it has been put into an indented section which the general 
reader is advised to omit. 


Since the method of scaling adopted is an арртахшиятап 
to the exact way of making the mean and standard der 
of the teachers’ estimates equal to the mean and standa 


ed А h pr 4 
deviation of the marks in the uniform examination, we may 
take it that 


Tes - Mo, +22(Te - Mg) 
Te 


It can then be shown that, for a single school, 


FQeO Te + FQaF Ta + (Coors + ©ов©те)7'теТа 
Vict ой, + 27летаетеоты}/{о$+ oba + 2гтеть©де© да} 
which would not normally 


For one of the schools 
'This is a fairly high сог 


, 
TTsTs = 


be equal to unity. 6 
in Mr Bain's sample 7тет = +96: 
relation, but, as we have seen, We 
cannot without other evidence say that it would not matter 
which we used for prognostic purposes. 

It can also be shown that 


OTe ET: + Ста?рта 


Trp = 
улт 


4 + 27отабтабта) 
ап 


OQef Ете  GQs PTa 
2 
КАСА + об 


From our other data we 
Tete Teta and тұту 


TTR = 


+ 27тетасоссоь} 


. : are not а very rigorous 
way of settling the question, but they lead to the following 
conclusions: 
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In the great majority of cases 7 would be higher 
than тұр but in exceptional circumstances 7747 could be the 
greater. 

For a single school of reasonable size rg would rarely 
exceed ray» by more than "03: the difference would normally 
be about -o2. 


Our conclusion is that the correlation with later success of 
teachers’ estimates scaled separately for English and Arithmetic 
would be higher, probably by about +02, than that for estimates 
scaled for English and Arithmetic together. A stronger case for 
the first method can be made out on other grounds, of which the 
most compelling is the need for satisfactory separate estimates of 
the pupil’s ability in the two subjects. 


SCALING TEACHERS? ESTIMATES on INTELLIGENCE Test RESULTS 


, The educational importance of this problem is much greater 
than at first sight appears. It lies in the fact that, if we could 
legitimately scale teachers’ estimates on the intelligence test, and 
if a combination of an intelligence test and teachers’ scaled 
estimates gave a high correlation with later success, we could 
avoid all the educational evils of uniform external examinations 
at the Qualifying stage. 

At this point we can deal only with the question whether the 
method would be legitimate, and we take first the simpler problem 
of the scaling of the total marks in English + Arithmetic. For this 
purpose we use as our standard of comparison the teachers! marks 
scaled by the soundest possible method, that is, оп Qualifying 
examination scores. With them we must compare the teachers’ 
estimates as scaled, say, оп ТО, which we shall call Тет. 

Mr Bain investigated this point with his sample of twelve 
Schools. 

Glancing along the rows of Table XVIII we can see that scaling 
оп IQ makes a considerable improvement, but not so great as 
that made by scaling on Q. Using the method previously adopted 
We compare the improvements in Table XIX. This is a Table 
Which we present with considerable regret. It shows that the 
method of scaling upon IQ cannot be regarded as sound. Even 
in the small sample of twelve schools injustice to the extent of 
Over 12 per cent. could be done to considerable numbers of pupils, 


and it seems unfair to use a system which has a known error of 
this size. 
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TABLE XVIII 
Teachers’ estimates scaled оп О and on IQ 
School Q Ts Th Jd 
1 556 551 549 673 
2 398 400 44-0 64:0 
3 575 569 62-0 672 
4 60-4 58-7 56:0 68-6 
5 564. 558 548 боа 
6 538 52-9 52:0 66:5 
7 55:6 53:6 541 754 
8 43:6 447 51-6 69:5 
9 441 43:8 49:9 “re 
то 378 379 432 бго 
II 34:6 340 - 41:5 59:0 
12 47 m 468 662 
AveraggeM . . 48-9 48-7 51-3 677 
Averaggg —. б 16:8 16:6 15-1 149 
TABLE XIX 


Teachers’ estimates scaled on О and on IQ: Average difference between 
school means and means of Qualifying examination marks 


"Type гелме Average difference Maximum error * 
per cent. per cent. 
T 69 E 
Ts1 3:8 т 
2:6 9 47 


* Maximum error —sum of largest positive and negative differences 


Mr Bain calculated the following correlatio 
estimates scaled on the Qualifying examina 
intelligence test for his group of 842 pupils: 


ns with teachers? 
ton and on the 
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Correlation of О and T (average) 4:836 
О and T (slumped) +787 
Q and Ts (slumped) +:883 
О and T$: (slumped) 4:857 
Ts and Те (slumped) + :971 
The most interesting of these correlations is the last. Despite 
the fact that the teachers’ estimates scaled in these two ways 
correlate to the extent of -971, we should, as we have seen, do an 
Injustice amounting to as much as 12 per cent. to certain pupils 
if we used the method of scaling on IQ. The middle pair show 
that the marks scaled on the examination correlate more highly 
with the examination than the same marks scaled on IQ. The 
first pair give another example of the way in which correlations 
are lowered when we slump together schools where there is a 
variation in standard of marking. 
‚ Another disadvantage of the intelligence test as a scaling basis 
is that we cannot obtain reliable estimates іп English and Arith- 
metic separately without assuming that the relative positions of 
the School means are the same in both subjects. The unsoundness 
of this assumption is clearly brought out by Table XV and by the 
following correlations between school means for the June Qualify- 
Ing group, comprising 36 schools: 
Correlation between school 
means 
ІО and Qe 4-88 
ІО and Qa +86 
Ое and Qa 4:89 
These are far from high enough to justify a scaling of English or 
Arithmetic on IQ. 


CONCLUSIONS 


We conclude, therefore, that the only satisfactory procedure is to 
scale the estimatesseparately for English and Arithmetic, and that the 
estimates should be scaled upon the results ofa uniform examination. 

Steps should at the same time be taken to ensure that all the 
teachers make their estimates on the same basis. The instructions 
should make it clear whether the estimates are to be measures of 
the pupil? attainments at the time of examination, Ог whether 
they are to take account of evidence of progressive improvement or 
deterioration; also whether allowance is to be made for ability, 
age and personal qualities. 


Lt | 


CHAPTER VI 


COMPARABILITY OF SUCCESS MARKS IN THE 
SECONDARY COURSES 


Tue Measure or Success 


To compare the prognostic value of the various measures used at 
the Qualifying stage we require a criterion by which they can be 
judged, and this should, strictly speaking, be the child's ability 
to profit by the secondary course. For scientific investigation, 
owever, we required one that could be expressed in exact 
quantitative terms; and we used the child's success, The two, 
we grant, are not quite the same: a child may profit much more 
from a secondary course than his success mark would indicate. | 
Yet, from the point of view of the child, the parent and the ! 
Education Committee, it is success that is the more important. 
Moreover, the pupil who consistently scores low marks in his 
secondary school examinations has much on the debit side of his 
‘ability to profit? account, no matter how much he has on the 
credit side in the way of response to cultural stimulus and social 
contacts, 5 
Accepting success as our criterion, we have still to determine 
what we are to take as its measure; and after careful consideration 


we decided to use the weighted average of the marks in all the 
subjects of the secondary course, 


Doubt as to the soundne 
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We were encouraged to persist in this view by the reflection 
that all the subjects would have fairly high inter-correlations, and 
by certain results obtained by Mr Robert Wilkie, B.Sc. Не 
prepared two lists of average first-year success marks for four of 
the senior secondary schools, the first being based only on the 
‘important’ subjects (Latin, French, German, English, History, 
Geography, Mathematics, Science), the second being based on all 
the e. Тһе average correlation between the two lists was 
985. 


IMPORTANCE or COMPARABILITY ОЕ Success MARKS 


This question of the comparability of the success marks is one 
of the most important in the whole technique of follow-up in- 
vestigations. Its difficulty may be illustrated by considering the 
June Qualifying pupils who proceeded to the junior secondary 
courses. They distributed themselves among different schools, 
different courses in the same school, and different classes in the 
Same course. We could not expect that the standards of the 
follow-up marks in the different schools or courses would be the 
same; therefore we had to deal separately with each course in each Ww 
School. This, as we shall see, raises serious statistical problems | 
if the attainment levels of the pupils in the different schools and 
courses vary, Furthermore, there is the possibility that the 
Standards of marking will differ even in the various classes in 
the same course in the same school. These difficulties can, we 
believe, be overcome. But far the soundest policy is to evade 
them by taking every possible step to ensure that the follow-up 
marks are comparable in the first instance. 


STEPS TAKEN To ENSURE COMPARABILITY оғ Success Marks 


This question was discussed at various meetings with the head 
teachers of the senior and junior secondary schools and we con- 
cluded that there was no feasible method of ensuring uniformity 
in standard of follow-up marks for different schools, or for the 
different courses in the junior secondary schools. All we could 
reasonably attempt was to-ensure that the follow-up marks would 
be comparable for the different classes in the same course in each 
junior secondary school, and that the marks would be comparable 
in the subjects common to the various courses in each senior 
secondary school. 

This was done by an extension of a system already in general 
Operation in many of the schools, whereby pupils in the different 
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classes of the same course all take the same terminal examination 
papers. To secure uniformity in the standard of marking many 
of the head teachers adopted the plan whereby each teacher, 
instead of correcting the papers of his own class, was assigned a 
certain number of questions in the papers of pupils in all the 
classes. From later inquiries Miss Young ascertained the exact 
-procedure that had been used in each school and course, and it 
was then found that it had not always been possible to adopt this 
System. In certain schools, although uniform papers were set, 
each teacher corrected the scripts for his own class, the standards 
being then co-ordinated by the head of the department. 

Тһе final position, in short, was that in most schools we had full 
assurance that the follow-up marks were comparable; in others 
we had a fair measure of assurance; but in a number of schools 
there was no guarantee that the same standard had been used. 


Снєск or COMPARABILITY or FoLLow-up MARKS IN THE 
Various Crassrs 

In the circumstances it was imperative that we should make a 
complete check to find which sets of marks should be excluded 
from our further calculations, and we did this in the following 
Way. If, for the purposes of illustration, we consider the pupils 
who elected a particular course in one of the junior secondary 
schools, these would be arranged in, say, four classes, A, B, C and D. 
For these four classes we have the Qualifying examination means 
which are comparable. Now both Qualifying examination and 
follow-up marks are measures of scholastic attainment made at no 
great interval of time; and we may therefore assume that if the 
follow-up marks were comparable the follow-up means would be 
in the same order as the Qualifying means and separated by 
similar intervals. 

If this assumption were sound, and if we were given the follow- 
up mean of Class A, we could calculate from the Qualifying data 
what the follow-up means of the other classes ought to be if the 
follow-up marks were сотрагаЫел 


1 This is done by a method similar to that-used when we transformed the 
teachers' estimates so that the mean and standard deviation of the estimates 


from a school were the same as those of the Qualifyi k ils from 
that school, The formulz are of the following а Е marks of the pupils fro 


Op, 
вМғ-АМ;- TMe -вМо) 


where АМр = теап follow-up mark for Class A, 
aMg=mean Qualifying mark for Class А 
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The first purpose for which we used this method was to test the 
validity of the underlying assumption that the Qualifying means 
give a sound guide as to the relative positions of the follow-up 
means. ‘This was possible because in many cases we knew that the 

. follow-up means were comparable, and we could compare the 
calculated means with them. Data for a case of this kind are 
given below: 


Class F means (per cent.) Z ғаш. 
Actual Calculated 
A 732 752 - i 
B 699 68-1 +18 29 
с 68-5 67:6 Tog 29 
D 66-7 64-7 42:0 2.8 
Е 57-3 61-6 -43 275 
Е 496 51-3 -17 51 


Тһе figures reveal the width of the gap that separates the 
best from the poorest class, a point whose statistical import in 
follow-up work we shall discuss presently. They also show that 
the calculated means agree very closely with the actual means, 
the differences being in all cases explicable by the fallibility of the 
measures concerned. 

"This method was applied to every class in every course in every 
School in all the three years of the follow-up, and the assumption 
stood the test in all cases where we had full assurance of the com- 
parability of the follow-up marks. We therefore felt confident in 
using the check as a criterion for discarding groups for lack of 
comparability. 

We give on p. 52 the data for a school in which we had not 
full assurance of comparability, but which we regarded as satisfy- 
ing the test. 

1 The probable error which is relevant here does not concern the variations 
which arise through taking different samples of pupils from a total population; 
it relates to the fallibility of the measurements. Knowing the fallibility of the 
Measures on which the calculated follow-up mean is based we can calculate 


its fallibility and the fallibility of the difference between it and the actual 
mean, 


Reference: D. Brunt, The Combination of Observations. Cambridge: 
Cambridge University Press, 1917. Р. 48. 

Тһе calculations are somewhat intricate and laborious, and in view of the 
number of cases with which we had to deal, and of the fact that great accuracy 
was neither possible nor necessary, the method used was to work out sample 
cases and interpolate by means of a graph. 
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F means (per cent.) 


СІ 4 2 
= Actual Calculated eae 
A 64-6 64-6 Т Ks 
B 61:3 57:6 “37 27 
c 5411 50-1 +40 2:6 
р 509 494 +155 30 


The following is a case where the test of comparability was not 


satisfied and the group of classes was therefore discarded from all 
further calculations: 


F means (per cent.) 
СІ а Edit. 
= Actual Calculated Pean 
A 63:0 63:0 % st 
B 63-4 53:6 + 98 24 
С 58-3 45-4 +129 24 


The amount of labour involved in this check was very great. 
We felt, however, that we were repaid by the confidence with 
which we could proceed with the further analysis of our data. 

If we leave out the top classes, where the calculated mean was 
arbitrarily fixed to be the same as the actual mean, the result 
was that in 69 per cent. of the cases the differences between actual 
and calculated follow-up means were less than their probable 
errors; 96 per cent. of the differences were less than three times 
their probable errors. "Тһе number of rejections was therefore 
not very great. 

А point which emerged from these calculations was that there 
is a general tendency for the follow-up means of the duller classes 
to bea little higher than one would expect on the basis of perform- 
ance in the Qualifying examination, The difference is not large, 
the average value being 2 per cent., but it is almost universal. It 
may be due simply to greater leniency in marking the poorer 


classes, or to a tendency to push these classes nearer to the limit 
of their capacity. 


COMPARABILITY оғ Forriow-up Marks ім THE 
Various Scuoors 


As has already been mentioned, we did not expect that the 
follow-up marks would be comparable between schools or between 
courses in the same junior secondary school, The latter was, 
indeed, no great disadvantage, for we wished to have separate 
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correlations for the various courses to see whether there were any 
significant differences. It may, however, be of some interest 
in connection with follow-up technique to show how the devia- 
tions in standard between school and school compare with those 
between class and class in the same school and course. We give 
below a comparison of actual and calculated follow-up means for 
е ] a group taking the boys’ technical course in junior secondary 
schools: 


F means (per cent.) 


School d plait. 
Actual Calculated 
A 53:9 53:9 E s: 
B 575 58-0 = 5 TA 
e 58-5 53:8 % 47 14 
D 483000571 -138 14 


Schools С and D differ in standard of marking by 18:5 per cent., 
and this holds almost exactly for all the courses in those two 
Schools. In the commercial course, for instance, the difference is 
18-4 per cent. у 

А comparison of actual and calculated follow-up means in senior 
secondary schools is shown below: 


F means (per cent.) 


School d Ёёаш. 
Actual Calculated 
A 64-8 64-8 Е Es 
B (617 69:3 -76 ГІ 
c 64:9 649 оо > 
D 64:6 66-9 -23 r3 


/ 
"The position is, therefore, that, thanks to the co-operation of 
the head teachers and staffs of the secondary schools, we could 
regard the follow-up marks of the classes in the same course in the 
Same school as comparable, but that the standards as between 
School and school, and as between the different courses in the 
junior secondary schools, could not be regarded as comparable. 


СНАРТЕК УП 
METHODS USED IN COMBINING CORRELATIONS 


"Tus is а tiresome chapter, devoted to certain points of follow-up 
technique, which the general reader will be well advised to omit. 


Tue PROBLEM 


Suppose we wish to find the best estimate of the correlation Тоғ» 
and that we have six schools for which the follow-up (F) marks 
ате not comparable though comparable within each school. 
Suppose also that the ability levels in the schools are different. 

One method of finding an estimate of тор would be to calculate 
а correlation-for the combined schools from a single grid, but this 
would involve an error caused by the differing standards of the 
follow-up marks. Another method would be to calculate separate 
correlations for the six schools and combine these by the usual 
х transformation. This, while avoiding the error of the first 
method, would introduce another, due to the differences in the 
ability levels of the schools. We shall call the first the error 
arising from differences in standard of marking, and the second, 
that arising from differences of the means, 


The problem is to find a method of combining the correlations 
which avoids both sources of error. 


Size or Error CAUSED ву DIFFERENT STANDARDS or M 


We can get a general idea of the error arising from differences in 
standards of marking as follows. If we have two large schools of 
the same size whose follow-up marks are on the same standard, 
the correlation for the combined schools would be 


Ху 
10,0, 
54 


ARKING 


Tor = 
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If the follow-up marks for one of the schools were all raised by 
4 per cent., the formula would become 


dof 


пс, y +. 
а оу 
4 


"Thus a small variation in standard would not have a serious effect, 
but a difference of 15 per cent. might lower the correlation by as 
much as ‘I. 

'The following correlations taken from Mr Bain's data give 
actual examples of the error in correlation caused by different 
standards of marking in one variate: 4 

English and 


English Arithmetic Arithmetic 


(N 2858) (N=853) (N —842) 
Slumped correlation with , 


teachers’ estimates " 4:766 TM 3:787 
Slumped correlation. with 

teachers’ estimates scaled 4:879 1:799 +883 
Loss in correlation . е 113 072 996 


Тһе differences in standards of marking are shown in Tables 
XIII and XIV, and we see that they give rise to a loss аара 
mately -1 in correlation. 


SIZE or Error CAUSED BY DIFFERENCES IN THE Means 


To illustrate the size of the error from the second source we take 
the six classes whose means are given on p. 51. In this case we 
know that the standards of marking are comparable, yet the 
difference of means is considerable, the average of the best class 
being 73-2 per cent. and that of the poorest 49:6 per cent. 

A simple diagram will show that there would be a considerable 
error if we used the average of the correlations. А correlation 
can be represented pictorially by an ellipse which is a horizontal 
section of the normal bivariate surface. Omitting certain technical 
reservations about scales and standard deviations, we may interpret 
а broad ellipse as a low correlation and a long narrow ellipse as a 
high correlation. From the data for the six classes we can find 
the centres and the shapes of the ellipses and the positions of their 
major axes, and plot these on a diagram. 

Fig. 6 shows the ellipses for the best and poorest classes. AB 
is the line of centres, and the other two lines are the respective 
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SUCCESS IN SECONDARY SCHOOL 
FOLLOW-UP 


Fic. 6.—Effect of differences in the means when the differences are 
nearly the same in both variates, 


Fic. 7.—Effect of differences in the means when the difference į hi 
J-variate is one-third of that in the X-variate, Бе 
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Major axes. The centres of the intermediate ellipses lie closely 
along АВ. . 

While a single class gives a broad ellipse corresponding to a low 
Correlation, the whole group would be represented by a much 
longer ellipse corresponding to a considerably higher correlation. 

The correct correlation for the complete group, however, is not 
always higher than that for the homogeneous sub-groups; every- 
thing depends on the position of the means. If the difference 
In means of the y-variate were one-third of the difference for the 
*-Variate, the position would then be that shown in fig. 7. 

It is clear from this diagram that the correlations for the whole 
8roup may be the same as that for the sub-groups, or even lower. 

To find the size of the loss in the case of these six classes, we 
calculated the correlation from a single grid and also by averaging 
the correlations for the separate classes. ‘The results are as follows: 


Average of correlations for six classes e +404 у 
Correct correlation for whole group from 

single grid , Р А i y . +619 » 
Loss through averaging . . ws 215 


The loss in correlation in this way is well known,! but it is 


Perhaps not generally realised that the difference could be as high 
"215 


Метнорв or avorpixc Error CAUSED BY DIFFERENCES 
IN THE Means 


Те above examples show that the error might be serious; and 
the question now arises whether it would have to be taken into 
account, say, in combining correlations for the senior secondary 
Schools, where the differences in means would be smaller. What 
We require is a method which will show whether a correction is 
Tequired, and will enable us to make it if necessary. 


USE OF A CORRECTION FOR RANGE? 


_It might be thought that we could overcome the difficulty 
Simply by correcting the correlations for the separate groups for 


1 Cf. for example E. F. Lindquist, Æ First Course in Statistics. Boston: 
Houghton Mifin Company, 1938. Р. 179. 
* Cf. G. H. Thomson, The Factorial Analysis of Human Ability. London: 
University of London Press, Ltd., 1939. Chapter xi. 
T. L. Kelley, Statistical Method. New York: The Macmillan Company, 


1923. P. 223. 
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range, but consideration of the possible arrangements of the 
ellipses will show that this would often be unsound. Further, 
if there were lack of comparability in both variates it would be 
impossible to obtain a satisfactory estimate of the standard devia- 
tion of the total population. 


SCALING THE FOLLOW-UP MARKS 


By using a process of scaling on Qualifying examination marks 
similar to that described in Chapter V, we can make the follow-up 
marks comparable and then calculate the correlation for the 
combined schools from a single grid. In Chapter VI we have 
given grounds for our belief that the underlying assumptions are 
legitimate, and another point in confirmation will be made at the 
end of the next section. А 

Instead of applying this method in the direct way, we found it 
better to use the following short-cut. 


CALCULATION OF FOLLOW-UP CORRELATIONS WITH SCALED 
FOLLOW-UP MARKS WITHOUT ACTUAL SCALING 


Since the scaling is done by equations, we can use these to find 
formule giving the values of the correlations with the scaled 
follow-up marks. 

If 7m denotes the correlation between intelligence score and 
follow-up marks for all the schools, A, B, C, D, . . ., when the 
follow-up marks are scaled on Qualifying examination marks, 


ХтдотдооаттғА + Элдар 
уо? + Ўпда?} V (Xn,064 + Ўтар?) 
where rma —correlation between intelligence score and follow-up 
mark for School A, 
na — frequency for School A, 
сол —standard deviation of Qualifying marks for pupils in 
School A, 
a=deviation of mean intelligence score for School A 
from mean intelligence score for all schools, 
p=deviation of mean Qualifying score for School A 
from mean Qualifying score for all schools, 
= denotes summation for all schools, А, В, C D,... 


70) = 


Formule 1 of this kind have some general interest. For instance, 


1 Тһезе formule can readily be put into terms of the correlations between 
the school means. 
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if we had calculated the correlations for the individual schools, we 
could find the true correlation for the combined schools. If the 
follow-up marks were known to be comparable, we could substitute 
Ova for oga, and so on. The formule also show at a glance the 
effect of combining correlations when the means are the same but 
the standard deviations different. Іп this саѕе, а= =. . .=р=4 
M ЕЯ 

As a test, of the formule and of the assumptions on which the 
Scaling system is based, we used one of them to calculate the true 
correlation for the six classes used as a previous illustration. From 
P- 57 it will be seen that the average correlation for the six classes. 
18 +-404 and that the true correlation is 4-619. With the 
formula the result is +-618. 


ÅPPLICATION or FORMULA TO DETERMINE NECESSITY FOR 
Correction ror Dirrerence IN Means ІЧ ForLow-ur 
CORRELATIONS ror COMBINED SCHOOLS 


‚Оп applying the formula we found, as we expected, that the 
differences between the average and the true correlations were 
small, being only about -оо7. Now, such a loss would be worth 
Saving if we could rely upon our data to that degree of precision 
and if we were absolutely confident of the exactness of the method. 

either of these conditions for statistical tampering with results 
was, however, satisfied; and we decided, with relief, that it was 
Sufficient to use simply the average values. 


ArPLtcAToN оғ FORMULA TO DETERMINE NECESSITY РОК. 
Совкксттом ror Dirrerence IN Means IN COMBINING 
CORRELATIONS FOR June Амр DecemBer Gmours 


At no point in our investigation did we wish to be in the position 
Of saying that our results were certain to be in error by some un- 

€termined amount. We therefore applied the formule to another 
case where we knew there was a difference of means. This arose 
in the combination of the Qualifying data correlations for the 
December and June groups. If we call the correlations between 
English and Arithmetic for the separate groups prna and jrpa, 
our problem was to find @+n7na- Іп this case there was lack of 
comparability in both variates, and we knew that the June group 


Was superior in. quality. 
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TABLE XX 


December and June Qualifying groups: Difference in intelligence 
and school attainment 


December group June group 
(N - 1,045) (N=1,940) M;-Mp 
M | ЕЕ M c 
IQ 98-47 12:53 IoI-80 14:50 3:33 
EQ* 98-22 13:65 101-67 1410 345 
АО * IOI-IO 1510 102-80 14:03 1:79 


* EQ denotes English quotient and AQ Arithmetic quotient. 


The differences in mean are statistically significant, and the 
question we have to answer is whether they are large enough to 
warrant the making of a correction in combining the correlations 
for the two groups. : ` 

The fact that both variates have to be scaled offers no mathe- 
matical difficulty, provided that we have a suitable basis for the 
scaling. In this case the only possible bridges are IQ, EQ and AQ, 
and we scaled I on IQ, Qe and Eon EQ, and Qa and A on AQ. | 

If we call the correlation between English and Arithmetic, 
when English is scaled on ЕО апа Arithmeticon AQ. 


» D +I E(EQ)A(AQ» 
the formula is found to be as follows: 


EnD родотовог”нА + Lnpps 
(прово + Lapp} / {Enp 040+ Эль} 
where р and q—deviations of the English means of the 

December and June groups from the mean 
of the combined group, 

sand t=deviations of the Arithmetic means of the 


December and June groups from the mean 
of the combined group. 


M+ TT E(EQ)A(AQ) = 


Formule for other variates can easily be written down once 
the principle of their construction has been grasped; and the 
results of their application are given in Table XXI. 

Again it appears that the error in our combined correlations 
arising from this cause is negligible. 
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TABLE XXI 


December and June Qualifying groups: Correction of combined 
correlations for differences in means 


5 
Correlation Correction 
I andE +002 

I and Qe 4-:002 
E and Qe -F:002 
А and Qa 4:001 

I and A —:003 

I and Qa —:003 

E and A —-o0I 

E and Qa --О0І 
Qe and Qa —+оо1 
Qe and А —+001 


ШР more than half of the cases the correction is negative, showing 
that the method of averaging correlations sometimes gives an over- 
estimate even when both means for one group are higher than the 


Corresponding means for the other. The explanation of this is 


given on pp. 56-57. 


VARIATION оғ CORRECTION WITH SIZE OF CORRELATION 
AND oF DIFFERENCE IN Means 


In the correlation тов the difference of means in both variates 
Was about 3-4 quotient points, and we concluded that a correction 
Was not necessary. It is of some interest to find whether it would 
have been necessary if the difference had been, say, double the 
amount, and also how the correction would have been affected 
if the correlation had been lower. The size of correction for 
oo sizes of correlation and of differences in means is shown 

elow: 


;, А Size of correction when difference 
Size of correlation Ear 


in means is 
34 68 
4:844 4:002 4:008 
4:600 +004 4:020 
4:300 +-009 4:036 
-000 +013 4:052 
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These results are not of general application, yet they show that 
the correction is larger for the smaller correlations and that its 
size rises steeply when the difference of means is increased. 


SreciaL DIFFICULTY IN CORRECTING FOR DIFFERENCES IN 
Means І FOLLOW-UP Investications 


In follow-up correlations we have seldom to deal with lack of 
comparability in the first variate. Our problem lies in the follow- 
up marks, and we have suggested that these should be scaled on the 
Qualifying examination. In doing so we are making the differences 
in mean the same for both variates in лор. It is, of course, very 
probable that they ought to be the same, but we are giving the 
benefit of the doubt to this one correlation, and thus putting it in 
а privileged position when we compare it with others. 

It was partly for this reason that we decided to make no correc- 
tion in our own follow-up, where all the corrections were small. 


CONCLUSIONS IN REGARD To F OLLOW-UP TECHNIQUE 


Our experience of this problem points to the following con- 
— clusions in regard to follow-up technique: 


Every possible step should be taken to ensure that the чне 
marks of the classes аге comparable within each school. Р І 
they аге not comparable they may be scaled on Qualifying 
examination marks. 

As far as the follow-up marks are concerned, it will normally 
be permissible to combine the correlations for the separate 
secondary schools by the z transformation, but before doing 
so the size of the correction for differences in means should 
be ascertained by the methods described above. 

The experiment should be so planned that scaling of the first 


variate is seldom necessary; and such scaling should be done 
only on a bridge in the same subject. 


CHECK THAT att CORRELATIONS WHICH 
BE REGARDED As 
POPULATIONS 


WERE COMBINED COULD 
DERIVED FROM Equatiy CORRELATED 


This was done by the x? methodi 
correlations involved was over 200, but it was usually obvious that 
the condition would be satisfied. The Statistical check was 

1 Gf. P. R. Rider, An Introduction to Modern Statistical Methods, Р. 106. 


Тһе total number of 
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applied in doubtful cases, and only in four of these did P lie below 
‘ol. Fortunately they fell in groups which were rejected on 
account of lack of comparability of follow-up marks, or concerned 
correlations with teachers’ raw estimates. 


VARIANCE AND CO-VARIANCE WITHIN AND AMONG Scuoors > 


The statistical reader will have noted the connection between 
the above formulæ and the methods of analysis of variance and 
co-variance, and it is possible that these methods could profitably 
be used in the study of the problems of this chapter. The variance 
for the five senior secondary schools is analysed below, the data 
used being the Qualifying examination marks: 


Degrees of Sum of Mean square 
freedom squares deviation 
Within schools . В 364 70,466 194 
Among schools . a 4 105 26 
Total . 4 368 70,571 


In this case z =} log, w = 1:00. 

The 5 per cent. value of z for m —24 is 877, and the рег cent. 
value is 1:317. For л; = co, the 5 and I per cent. values are 864 
and 1-300, The obtained value for the significance of the variance 
among schools is therefore between the 5 and г per cent. points. 

The largest variation among schools in all our follow-up data 
was that in the commercial course for the December group, and 
it was only after some hesitation that we decided to make no 
correction for differences of means even in this case. It was, 
however, a small group, and we felt that the absence of correction 
would make little difference in the final results. Below we give 
the analysis of variance and co-variance for this group of four 
junior secondary schools, the data being for the correlation 74р: 


Degrees of Sum of Sum of Sum of 
freedom squares products squares 

| А Е 
Within schools . 120 > 13,277 5,824. 10,847 
Among schools . 3 1,674. 712 ` 804 
Total „ - 192 14,951 6,536 11,651 


‚ 
Тыз shows а much greater variation among schools, but even 
here the regression among schools is not significant. 


CHAPTER УШ 
COMBINATION OF CORRELATIONS 


Tue senior secondary group was distributed over five schools, 
while the main body of the junior secondary pupils went to four 
large schools, where they split up into three courses. | 
"This complication of groups entailed the calculation of approxi- 
mately 1,000 follow-up correlations, which have a somewhat 
oppressive effect if presented in the mass. We therefore give 


merely sample Tables to show the stages by which we arrive at the 
final combined results. 


SENIOR SECONDARY SCHOOLS 


ORIGINAL TABLES ror SEPARATE SCHOOLS AND YEARS 


TABLE XXII 


Sample Table of correlations for a senior secondary school 


Бо ка ifi if, Ёз ay ny aF ay 
I 715 714 :68о 710 “653 :639 
IQ 696 | 694 | 703. | -707 | «599 | -610 
E :580 -616 :586 “бо? *581 515 
e Т ЧОО Ср) Өз | „оў 
Ое `743 `732 731 -760 -636 -584 
Qa 662 669 695 “710 -665 -616 
S 707 725 :799 726 -694 -614 
Q “773 7771 :785 -809 716 -660 
T "6 | 68; | 6% | -688 -632 572 |, 
ТІ a os, zt 774 654 652 


Even in this single school the correlations аг 
and many of the final results cast their sh. 
; a 


€ reasonably steady, 
adows before, For 
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instance, Ts always shows a higher correlation than Т; the correla- 
tions for Qe are always higher than those for E. This fact, that 
most of our results held not merely in the final combined Tables 
but for every school and for every year, added greatly to our 
confidence in them. 

Only a very small number of the entrants to the senior secondary 
schools were from the December group, and these were omitted. 


First CONDENSATION: COMBINATION оғ SCHOOLS 


We had five Tables of the above type, and the next step was to 
reduce them to a single Table by combining the correlations for 
the five schools. 


TABLE XXIII 


Correlations for senior secondary schools combined 


Correlati 
iae Fy ify 1% 1ЁАУ Fav Fav 
I 437 | yor | 6% | ло | 643 | 646 
IQ gos | Өт | 695 | бо | 647 | “бю 
Е *594 | +576 | 593 | 595 | 53+ | 7529 
А -736 -697 -680 -690 “629 *567 
Qe 7766 | 733 738 749 | 631 593 
Qa ‘671 -672 -696 -686 625 `597 
5 745 7и “715 717 655 615 
Q +802 “781 *800 "810 714 “673 
T: “705 674 -680 -659 616 80 
Ts зе va s 737 677 640 
Average r 78 :691 *699 704 637 -608 
N 367 | 359 | 367 | 368 | 333 | 259 
во ғ” “ . 13-83 12-97 11:15 


Correction oF SECOND- AND THIRD-YEAR CORRELATIONS 
FoR RANGE 


The differences between the correlations for the three terms of 
the first year are not significant statistically, so we may conclude 
that, on the average, the results do not establish a fall during the 


first year. 
3 
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The slight rise from ,F, and ,F; to ¿Fay might also Бе due 
to the fluctuations that are to be expected with relatively small 
numbers. There is, however, a mathematical explanation, in that 
the correlation with an average will usualy be less than the 
average of the correlations with the elements which make up 
the average.i 

We can also explain part of the fall from year to year. There 
is little or no leakage in the first year, but thereafter the numbers 
fall rapidly, due largely to the dropping out of weaker рр 
Тһе narrowing of the range of talent, indicated by the standar 
deviations at the foot of Table XXIII, will cause a fall in the 
correlations. Ж 

We can correct for this narrowing of range,? and the соге) 
then approximate to what they would have been if al} the рар” 
had gone on to the end of the third year. This has two advantages: 
it makes the correlations for the various years comparable, thus 
removing one obstacle to their further combination; and it 
enables us to show the true fall in predictive value from year to 


year. (See Table XXIV.) 


Finat CONDENSATION: COMBINATION OF CORRELATIONS 
FOR THE THREE YEARS 


The main aim of this chapter is to show the successive stages 
by which the correlations were condensed to single values for 
each measure, and further discussion of the fall from year to 
year is deferred until the corresponding results for the junior 
secondary schools are available. Until then, we shall also provision- 
ally regard the other conditions for the combination of the correla- 
tions for the different years as satisfied. (See Table XXV.) 


1 The average of the correlations with I is 
Tir, +7, + TIP, 


The correlation with the average is 
TY, +F +P) 


ТІР; ri, +r, 
УЗ алив, алев, алтар) 
if F4, Е, and Ез are given equal weights. 
Since 7r Fy TFF, and ғғ,р, will normally be less than i, е зесопа method 
would give a higher result than the first. е 
2 Cf. р. 57- Тһе narrowing of the range in follow-up marks was regarded as 
consequential upon the narrowing of the range in the first variate, 


which equals 
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TABLE XXIV 


Correlations for senior secondary schools combined: Second- and 
third-year results corrected for range 


Correlati 
e am Fay olay al ay 
I 710 "667 724 
IQ -690 -671 718 
E '595 | :559 | бп 
A *690 “653 -648 
~ Qe 749 | 655 | 674 
Qa :686 -649 :677 
5 “17 | 679 | -695 
[e] “810 +736 748 
T 659 +640 +661 
Ts 737 “701 718 
Averager | -704 -661 -687 


TABLE XXV 


Final order of predictive value of measures for success in 
senior secondary schools 


Order сте Final combined correlation 
with success 
Qe 4:699 
Qa 4671 
A +-666 
Е +-587 
Q 2770 
Ts ++720 
I ++700 
S 4-:698 
IQ +-691 р 
T 3-653 
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CORRELATIONS FOR BATTERIES OF THE ORIGINAL MEASURES 


Few Education Committees would now, we hope, use a single 
test or examination for purposes of guidance or selection. They 
would be more likely to use a pair, like IQ +Q, or a battery of 
three, like 1Q+Q+Ts. We have now, therefore, to find the 
order of merit for all the possible batteries,! and for the reasons 
offered in Chapter X we gave all measures equal weights. 

The final correlations for batteries are given in Table ХХХП, 
along with those for the junior secondary schools. 


JUNIOR SECONDARY SCHOOLS 


The results for the junior secondary schools are much more 
complicated and the number of original correlations much larger. 


ORIGINAL TABLES ror SEPARATE SCHOOLS AND YEARS 


TABLE XXVI 


Sample Table of correlations for a junior secondary school 
(Boys’ technical course) 


December group June group 
Correlation 
with 

Fay sFay | iF ay 2ЃАу 
І :606 610 -670 718 
IQ 569 | 754 | -650 673 
Е 419 | 415 536 | 5534 
A -627 "664. :548 +529 
Qe :632 -609 709 | -610 
Qa 48 | 5596 | -545 | +520 
ЕБ “бі -63о 517 554 
Ts :708 “697 +806 -728 


“There were twenty such Tables. 


3:19 references need be given on the methods used here; they will be found 
in any text-book of statistics. 
The following is a sample formula: 
ТІР +7 ОР --7ар + 7твр 
: V/Á4 +270 + 2718 + 2711s лов + d mere. 
АП the inter-correlations of the measures E therefore necessary, and these had 
to be obtained separately for the senior and junior secondary Broups, 


T(I--Q--8--TE)F = 
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We had four to six schools, three courses, and December and June 
groups. Only for the commercial course was it possible to obtain 
correlations for all three years of the follow-up; for the boys and 


girls’ technical courses the numbers in the third year were too 
small. (See Table XXVI, p. 68.) 


First CONDENSATION: COMBINATION OF SCHOOLS 


The first step is to combine the schools after ascertaining that 
it is statistically legitimate to do so. When this is done we get 
the results shown in Table XXVII (p. 69). 


Seconp CONDENSATION: COMBINATION OF DECEMBER AND 
Juse Своорѕ 
The December and June groups can be regarded as equally 


correlated populations, and we therefore combine them, giving 
the results shown in Table XXVIII. 


TABLE XXVIII 


Correlations for junior secondary schools: Schools combined: 
December and June groups combined 


«НЕЯ CENE n up qui алы cg: NC 


> ns Р 
Pr : 1 Едіге 1 Commercial 
- technica. technica 
Correlation course 
E course course 
with 


Fay oF ay 1Ғау oF ay ay al av oF av 


16 | +589 | -631 | -583 | -529 | 519 | 568 


IQ 595 | -595 | -609 | -624 | -520 | 502 | 557 
E 523 478 509 *516 461 428 457 
A 468 | -410 | -573 | -521 | 511 | 517 | 498 
Qe 632 *597 604 *562 595 :555 561 
Оа sor | +475 | -531 | -418 | -491 | -410 | 468 
5 628 *561 633 605 581 566 -579 
Q 667 -631 661 -570 “645 -609 -606 
T :578 | 593 | бо | -528 | -531 | -536 | -570 
Ts -661 649 -640 -611 -640 -627 -599 
Averager | 587 | :558 | "599 | -554 | -550 | :527 | 546 
N 768 533 | 541 333 | 342 | 294 | 146 
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CORRECTION or SECOND- AND THIRD-YEAR CORRELATIONS 
FOR RANGE 


The following correlations are those that would have been 


obtained if the whole of the first-year group had completed the 
course: 


TABLE XXIX 


Correlations for junior secondary schools: Schools combined: December 
and June groups combined: Second- and third-year correlations 
corrected for range 


Boys’ Girls’ 


ч 2 Commercial 
technical technical 
р course 
Correlation course course 
with 
Pay al’ ay Fay oF av ay al ay sFav 
I 616 | -621 631 | -606 | -529 | 515 | 550 
16; 595 | -627 | -609 | -647 | 520 | 498 | 558 
E 523 | 509 | -509 | 539 | 461 | чаз | 434 
a 468 | 439 | 573 | S44 | $11 | 5813. | 477 
Qe 632 | +629 | -бо4 | -585 | :595 | 5551 | 542 
| Qa от | 506 | -531 | 439 | 491 | 405 | "45 
S 628 | -593 | -633 | 628 | :581 | -563 | :562. 
[e] 667 | -662 | -661 | 593 | -645 | -607 | :591 
T: :578 | -625 | -604 | 551 | +531 | :532 | +552 
Ts | -661 | -680 | -640 | -634 | бю | -625 | 583 
Average r | 587 | 59 | -599 | +577 | e550 | 5аз | +527 
Average r 
uncorrected 
for range | 487 | 558 | -599 | -554 | 559 | +527 | 546 
N 768 533 541 333 342 294 146 
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Тнікр CONDENS. 


Correlations for junior secondary 8 
December and June groups com 


ATION: COMBINATION OF CORRELATIONS 
FOR DIFFERENT YEARS 


TABLE XXX 


chools: Schools combined: 
bined: Years combined 


Correlation with Boys’ technical | Girls’ technical | Commercial 
i course course course 
I “618 -622 +528 
IQ -608 -624 “515 
E “517 521 442 
А 456 +562 +506 
Qe -631 -597 -569 
Qa 503 497 451 
5 614 -631 571 
Q -665 +636 -621 
T 598 585 535 
Ts -669 :638 :624 


Еоовтн CONDENSATION: COMBINATION OF CORRELATIONS 
FOR DIFFERENT COURSES 


TABLE XXXI 


Final order of predictive value of measures for success in 
junior secondary schools 


Order of merit 


Final combined correlation 


with success 


—— a 
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We shall see in the next chapter that the statistical requirements 
for this combination are satisfied; but even if they were not, it 
-would still have been desirable to make it, for only thus can we 
arrive at a final general order of merit of predictive value applicable 
to all post-primary courses. (See Table XXXI) 


SIMPLIFICATION OF TABLES FOR BATTERIES BY Omission OF 
CORRELATIONS WITH INTELLIGENCE SCORE AND TEACHERS’ 
Unscatep ESTIMATES 


„Our next step is to find the correlations for batteries; but, in 
view of the results already given, there is no point in retaining 
data for the teachers’ unscaled estimates, which should not be 
used for prognostic purposes. 

After making this simplification we are still left with 23 different 
combinations of the fundamental measures, none of which includes 
both intelligence quotient and intelligence score. If we could 


[чишу the omission of either IQ or I we could reduce the number 
0 15. 


COMPARATIVE PREDICTIVE VALUE OF INTELLIGENCE SCORE AND 
INTELLIGENCE QUOTIENT 


d The correlations for all the combinations, one of which includes 
шесе score and the other intelligence quotient, were cal- 
culated, and the differences are shown below: 


Correlation for battery with I, minus 
correlation for battery with IQ 


Battery t 
Senior secondary Junior secondary 
schools schools 
I. . . s 4-009 +:007 
EO u^ „> 4014. 4-002 
I+S . . i +-008 -000 
LETS . К 3 — -003 = -003 
1+О+5 . . +-009 ооо 
TOTS @ 5 +-004 —-001 
1--5 +Ts ч қ +оо —-003 
I+Q+S+Ts. Л +004 = 001 


_ The average difference is -004; none of the differences is 
significant statistically; I is higher in nine cases, IQ in five cases, 
25 
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and in two cases the correlations are equal. , Hence, so far as our 
data go, the two may be taken as equal in predictive iem M 
This point has a bearing upon the question of diis oe E 
and also upon the difficulty created by the limited supply o um 
with reliable norms. If we have only three or ш а b di- 
to choose, we cannot avoid repetition and the possibility о еа 
dates preparing forallofthem. Since, however, 5 
is as good as intelligence quotient, we may use an unstan 7 ae 
test and thus widen our range of choice. Of course we ne mE 
then put the results in the familiar and easily a 
IQ; but we feel that, in view of the lack of comparability bet Ma 
IOs determined by different group tests, this might be an 
vantage rather than a disadvantage. А А ere 
With some hesitation we decided to omit mais we 
rather than intelligence quotient. This is not so sound statistically, 


Я : В re in 
for we mix scores and quotients; but the use of ТО is mo 
conformity with current practice. 


E ES 
CORRELATIONS ror BATTERIES OF THE ORIGINAL MEASUR 


The correlations for 
those for the senior s 
that the range of tal 


the junior secondary schools are lower ве 
econdary schools. Now we should S 

ent in the intellectual élite that enters E 
senior secondary schools would be narrower than that in the 
junior secondary courses; and that would no doubt be so арг 
In 1935-1936, with no strict selection, the standard deviation o 

the Qualifying examination marks for the senior secondary group 
was considerably larger than that for the junior secondary groups. 


We therefore applied a correction for range to the correlations 
for the latter, th 


us permitting the comparison shown in Table 
XXXII. p s paris 


The agreement is remarkably close, both in size of correlation 
and in order of merit. It is, of Course, not necessary that this 
should be so; and the differences are not necessarily due to 
sampling or experimental errors. It jg quite possible that the 
order of predictive value may be different for the two types of 
school. 


We started with nearly 1,000 correlations, These have now 
been condensed to 8, those for IO, Q, S and 'Ts, and then expanded 
to 30. In performing this condensation we may. тое have 
taken a little statistical licence, but "Table X ТЇ as der of 
respect in consideration of the labou ^ 


T involved in its preparation. 


COMBINATION OF CORRELATIONS 75 


TABLE XXXII 
Predictive value of batteries for senior and junior secondary schools 
= 
Senior secondar: unior secondary 
Battery schools дї schools * 4 
IQ+Q+Ts . қ -804 -806 idus 
1Q+Q+S+Ts . +800 -799 +001 
О 5-7. . 790 784 +-006 
IQ +Q. " " -786 -774 +012 
Q «Ts T | 783 780 4-:003 
10-0-5 . 3 783 773 +оо 
IO«S «Ts . à 782 793 ol! 
1Q+Ts 0. . 779 :795 -016 
HR a s 774 753 son 
Q * . . 770 -738 4:032 
DEPT „ 764 775 -оп 
10-5. " 4 736 741 —+005 
Тз . è y +720 741 —-021 
5 . . i +698 +705 —:007 
IQ . . з *691 -689 +002 


* Corrected for range. 


CHAPTER IX 


PRELIMINARY DISCUSSION OF THE FOLLOW-UP 
A CORRELATIONS 


; ; í eneral 
Iw this chapter we draw attention to certain points of g 


È ‚ fustify certain 
interest arising from the results just presented, and justify 
of the combinations we have made. 


: LIFYING 
Is THERE A FALL IN THE Prepictive VALUE or THE QUA 


NDARY 
.Measures ім tHe Later Years or THE SECO 
Courses? 


is a fall, 
In the preceding chapter we have shown that there is a ive 
but that this can be partly accounted for by the eic 
narrowing of the ranges of the groups. We can now pres 


“опе Table all the data bearing on this question: 


TABLE XXXIII 


Average follow-up correlations for the first three years of the 
secondary courses * 


N r 
Course 


Senior secondary , M 368 -704 -661 -687 
Junior secondary : 
Boys’ technical 


768 “58 +58 
Girls’ technical , : 541 2 E ЫН 
Commercial. . E 342 “550 -523 "527 


* Тһе correlations for the second and third 

When we apply the statistical tests 1 we find that none of the 
differences is Statistically significant, 

* For the senior secondary and com; 


i i he 4? test is the most 
conveniente) Eoft the senior ok Courses, where the differences are 
221 arly 1-2: P is therefor b : 5 
greatest, у is nearly i е about -5. For the girls’ technical 
course the ж difference is 034: SE=.019. | girls’ tec 
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years have been corrected for range. 


mercial courses (| 


— AM 


PRELIMINARY DISCUSSION OF CORRELATIONS 77 


While the differences in average correlations are not significant, 
it is possible that those for the individual measures might be. 
This was tested for the four main measures, IQ, О, $ and Ts, and 
it was found that, on the whole, the correlations might be reason- 
ably regarded as coming from equally correlated populations. 

" These results give some statistical justification for the combina- 
tion of the correlations for the different years; and, while they do 
not rule out the possibility of a real fall from the first to the third 
year, they show that it is unlikely to be much in excess of “02. 
The point is of great importance, for if the fall had been consider- 
able, we should have had to recast all our ideas as to the possibility 
of selection at the Qualifying stage. Measures which give a good 
Prediction of success in the first year only would be of little value. 


Rise ін Рвврістіуе Varur or IQ iw LATER YEARS 
or Courses 


An examination of Tables XXIV and XXIX gives little or no 
evidence of change in the relative position of I and IQ in the later 
years of the courses; but it does confirm the view that intelligence 
will tell in the end, and that the IQ rises in predictive value 
relative to the other measures as the years goon. In every course 
the correlation of IQ with F rises from the first to the last year, 
while the correlations for § and Q always fall. The differences 
are not great, but their consistency adds to our confidence in their 
existence. A comparison of correlations for the first and last years 
for all secondary courses is given below: 


Average rise or fall from 


Measure first to last year 
IQ Rise -029 
Ts Fall -016 
S Fall -020 
Q Fall -047 


The puzzling fact that the correlations are higher for the third 
than for the second year may be due to chance fluctuations or to 
the correction for range. ‘There may, however, be other reasons. 

he pupils may work more consistently up to their capacity, and 


1 The most doubtful case in the group for the senior secondary schools is 
that of О, where 72—7-3, and P lies between -o2 and -о5. Inthe junior secondary 
group the only difference which approaches significance is that for Q in the 
girls’ technical course. Неге the x difference is -112: SE=-o61, 
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f 
there may be increased steadiness in the teachers үзү. 
marking as the Leaving examination approaches. E ent Ld 
pupils who have left comprise for the most part Р lee 
terested in the secondary course, and in this group E ec bably be 
between Qualifying attainments and success would pro 
low. . . Я 

This tendency for the third-year correlations to be ae ule 
those for the second makes it difficult to give an exact em uti 
to the significance of the rise in the follow-up corre ic poe 
ТО relative to those for the other measures, as in dare if we 
second is the last year, and in others the third is the yp bead 
consider Q, the fall from the first to the third year in = бер 
secondary group is just significant; 1 and while the Pa 5 
is not in itself significant, the difference in relative а= е 
апа О in the last year is decidedly so. We may there pe їп 
аз very probable that in the later years of the course 
predictive value relative to the other Qualifying measures. 


ATER 
Dors tur ORDER оғ Prepicrive VALUE CHANGE IN L 
YEARS оғ THE SECONDARY CouRsE? 


TABLE XXXIV 


Changes in order of predictive value of the measures in the 
first three years 


Place in order of predictive value 

Junior secondary 
Measure | Senior secondary Boys’ Girls? Commercial 
technical | technical course 

course course 
Year 2 3 1 -2 1-5% 2 

о Te ШОТ TED OI naf 
Ts 2: 12) „2 DET 2 2 Zr 2 
5 2% dm 3 4 8*3 зв 
о 4 4 3 4 3 a Кт 4, 4 4 


* Cf. footnote, p. 77- 


4 
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On the whole the measures retain their places with reasonable 
consistency, especially when it is remembered that a change in 
order may be due to a very small change in the relative sizes of 
the correlations which determine іє. As we should expect, IQ 
Shows a faint tendency to improve its place in the later years. 

‚ The answer to the question therefore appears to be that there 
is no clear change in the order, but that there is a change in the 
spacing. 


Are THE CORRELATIONS HIGHER FoR SOME COURSES THAN 
ron OTHERS? 


\ 
From the figures given in Table XXXIII it would seem that 
the correlations are highest for the senior secondary courses, very 
nearly equal for the boys’ and girls’ technical courses, and lowest 
for the commercial course. But when we take account of the 
relative ranges, we find that there is no evidence of any difference 
between the senior secondary, the boys’ technical and the girls’ 
technical courses, There is, however, a statistically significant 
difference between these and the commercial course,! and once 
again our confidence in this curious result is increased by the fact 
that it appears not only in the averages, but separately for the 
December and June groups, and also for each separate year. 


Dors тнк Orper or PREDICTIVE VALUE VARY FROM COURSE 
TO Counsr? 


TABLE XXXV 
Order of predictive value of the measures for the different courses 


Junior secondary 
Mea Senior Э 
Ае secondary Boys’ Girls’ ‘Commercial 
technical technical 
Ts 2 I 1 I 
Q I 2 2 2 
5 3 3 3 3 
IQ 4 4 4 4 


1 If we take the first-year correlations for the girls’ technical and the com- 
mercial courses the ж difference of the average correlations is :073: SE —-023. 
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Our answer is, therefore, that the predictive value of the 
Qualifying measures is the same for the senior secondary, the | 
and girls! technical courses, but slightly lower for the commercia 
course. 

Тһе order is even more consistent than for the different years 
of the same course; and the only notable point is that О takes first 
place in the senior secondary course, while T's gives the cuu 
correlation for all the junior secondary courses. If we take the 
separate senior secondary schools and years we have fifteen aie 
parisons between Ts and Q, and in all but three of these Q takes 
first place; in all but four of fourteen comparisons for the junior 
secondary schools Ts takes first place. : b 

The answer to our question is, therefore, that there is a prob- 
ability that Q is better for predicting success in the senior secondary 
Schools than Ts, but that otherwise the order of the measures 15 
the same in all types of secondary course. 


„=. 


CHAPTER X 
THE WEIGHTING OF MEASURES 


NOMINAL AND Астолі WEIGHTS 


Wz first warn the general reader against two sources of mishnder- 
standing: 


If Qe, with a possible mark of 200, and Qa, with a possible 
mark of 100, are combined to give Q; or 
If the possible marks in Qe and Qa are roo, and if, before 
combining, we double the English marks; 
in neither case are we necessarily giving English twice the weight 
of Arithmetic. 

Тһе reason is, that the actual weights depend upon the relative 
dispersions of the marks combined. If oy and ол are the standard 
deviations of the English and Arithmetic marks, and if, before 
combining, we multiply the marks by the nominal weights, Wy 
and ХУ, (giving the combined score W;E +W,A), then the actual 
weights given are Wyoy апа Waca. 

When we combined Qe and Qa to give Q, the weighting was not 
I$: 1, buta complex one depending upon the standard deviations, 
and specific to our own examinations. It was for this reason that, 
in finding correlations for the batteries, we used the simple standard 
system of equal weights. 

„Ме must now consider the questions, whether a better pre- 
diction would be obtained by using a different weighting system, 
and whether there are any general rules in regard to weighting 
which Education Committees should follow. For this purpose 
We use a statistical technique 1 which enables us to find the weights 
giving the highest correlation with F, called the best weighting, 
and the size of the correlation for the best weighted battery. 


WEIGHTING or ENGLISH AND ARITHMETIC 


The problem is to find what weights should be given to English 
and Arithmetic in combining Qe and Qa to give Q, or in combining 
1 G. H. Thomson, “On the Computation of Regression Equations, Partial 


Correlations, etc.," British Journal of Psychology, vol. xxiii, Part I, July 1932, p. 64- 
8т 
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E and A to give S. То solve it Dr McIntosh and Miss Young 
calculated the best weighted correlations for the combined senior 
secondary schools and found the differences between these and 


the equal weighted correlations. The improvement effected is 
shown below: 


Com Difference between best — 

Боп weighted and equal weighted , Best weighting 
correlations with F 

ifi iF, 1з Pay Fay Fy 

Qe+Qa -005 -005 соот -оо2 сооз 3 

2 


43 
E «А -007 -006 -оо7 -007 -0o09 2: 


woe 
ә м 
он 
[3 
оз 
- 
N 


22 
:3 

Тһе differences are, of course, all in favour of the best weighted 
correlations, and they show that the improvement resulting from 
best weighting is very small. When we take this in conjunction 
with the facts that the best weightings are different for the tests 
and for the examinations, that they vary even for the same pair of 
examinations, and that the best weightings given above are specific 
to our particular tests and examinations and would not necessarily 
hold for others, we may conclude that the weighting of English 
and Arithmetic in Qualifying tests or examinations need not be 
attempted, 

Barrerres—Besr WEIGHTING 


The improvement effected by the best weighting of batteries, 
as shown b 


y the increase in the follow-up correlations, is as 


follows: 
Senior secondary Junior secondary 
Battery schools schools 1 
Improvement Weighting Improvement Weighting 

I+Q+Ts . "004 5:7:3 +002 6:5:8 
I +Q+S+Ts 008 6:9:1:4 оо git £8 05 
ОСО , +005 4:7 -003 2155 
I+Q+S 013 6:11:1 "007 8:3:12 
Qro +Ts . *004. Ta 3:4: :003 3:4:6 

Mr 2 +003 52427 -009 11:5:16 
Q+Ts . . "004 4:3 -000 ІЗІ 
I+Ts . . “000 5:6 -002 Б 
О+5 . 3 "008 Bins: -003 1:2 
Sls .- 2 "oor 9:11 55 2:3 
LS y 2 -000 Ісі -000 4:5 

1 Тһе calculations for the 


junior second. 
Miss M. Mitchell, M.A. ary schools were made by 
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b The improvement is very small. It would be well worth making 
if we could assign systems of best weights for various batteries 
which would be generally applicable, but the above results show 
that this is impossible; even with the same tests there are striking 
variations between the senior and junior secondary groups. 

Mr А. S. Robertson, M.A., using a group of four senior secondary 
schools, found the following variations in best weighting for the 
same tests and the same group of pupils: 


Battery Best weightings for correlation with 


non ifa n Fay ny 
loeum $:12:7 3:13:11 4:11:9 4:12:9 6:9:13 
Q MESA . 9:4:9 6:5110 8:4:9 8:4:10 ніп 
wp rt) 13:6:5  13:7:3 11:8:5 1217:5  9iI2:2 
ET . 9:10:14 9:11:12 9:10:14 9:10:15 8:11:11 
| +Ое+Оа+Т 2:4:2:2 1:5:3:1 1:4:3:2 1:4:3:2 I 3:5:8 
О+Е+А+Т . 3:2:3:4 2:2:3:4 3222324 2:2:3:5 I 2:4:4 


, Some of Mr Kidd's results show that the best weightings for a 
Біуеп battery vary considerably from school to school: 


Best weighting for correlation of 


School battery 1+Q with 
if Ёз Fs Fav 
A 23 1:2 1:4 rm 
B 1522 2255 2:5 1:8 
с 22095 23 3:4 2185 


In view of these results we find it impossible to lay down a single 
principle even of the most general character. Since, in addition, 
the gain by any system of best weighting is very small, we recom- 
mend that Education Committees should simply take an ordinary 
average of the results of the tests which they employ. The time 
and labour involved in the calculation of averages with complicated 
газа would be better devoted to the scaling of the teachers’ 

arks, 


CHAPTER XI 
CORRELATIONS WITH THE BEST BATTERY 


Waen no follow-up data are available as a criterion, a rough y 
of the relative values of the various measures can be obtained by 


- assuming that the battery of all the tests is likely to be the best, 
and calculating correlations with it. 


Тһе method has too much of the character of a vicious circle 
to command great confidence, but since we have a follow-up 


TABLE XXXVI 


Correlations of batteries with best battery IQ +Q + Ts 
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criterion our data allow us to make a test of itsvalidity. For this 
purpose we used the best battery, IQ +Q 4 Ts, rather than the 
complete battery.1 2 

Our interest lies in the way in which the order so obtained 
compares with that of the follow-up correlations. "Тһе latter, 
Which are given in column (4) of Table XXXVI, are the average 
correlations for senior and junior secondary schools; and to make 
the comparison easier we have reduced those with the best 
battery to the same scale. "This was done roughly by equating 
the highest and lowest values and placing the intermediate points 
by means of a straight-line graph. Mos 

"Те conformity between the two orders is remarkable; it is, in 
fact, much closer than that between the follow-up correlations 
for the senior and junior secondary schools. The average. 
difference, without regard to sign, is -004, as compared with 
‘orl for the differences in Table XXXII. 

These results give some justification for the belief that the >» 
method has evidential value; and, in view of the labour involved 
in large follow-up experiments and the time that must elapse 
before results are available, all ways of making legitimate short- 
Cuts are important. Methods of this kind have, of course, serious 
limitations, If a new battery were prepared we could find its 
correlation with ТО +О 4- Ts. If this came out to be :996 we 
could say that it was probably a very good battery, but we could 
Not say whether it would give a higher or lower correlation with 
Success, If, on the other hand, the correlation were “4, 1t would 
Probably be a poor battery. 


! The correlations were not calculated directly from grids but by means of 
formule of the type mentioned in the footnote on p. 68. 


СНАРТЕК ХП 


PROFILES: A SECOND METHOD OF DETERMINING 
THE PREDICTIVE VALUE OF MEASURES 


Eacn child at the Qualifying stage sets a problem: by the end of a 
follow-up inquiry we know the correct answer. We can therefore 
study any suggested procedure of selection, using as our test the 
number of cases in which our answers would have been wrong. 


Limitations oF THE CORRELATION TECHNIQUE 


The correlation technique, which we have used so far, has many 
advantages, but it involves an immense amount of labour and 
has serious limitations. On the basis of the correlation of the 
various batteries with success we should expect to make the fewest 
mistakes in selection by using IQ +Q + Ts, but this is not absolutely 
certain, The correlations may not be uniform throughout the 
range: a battery may be highly selective at the top of the scale and 
less selective at the border-line. Further, the evaluation of a 
battery by correlation ignores the possibility of improving its 
selectiveness by considering, in border-line cases, the way in which 
the total score of a pupil is made up. А 

Another serious limitation of the correlation technique is that, 
by itself, it gives us no information on the very important practical 
problem of fixing the pass-mark, a point which we shall postpone 
to a later chapter. 

For these and other reasons given later we decided to work over 
our data again, using an entirely different technique which throws 


light on the blind spots of the selection problem left by the correla- 
tion procedure. 


PROFILES 


For this alternative method it was necessary to have the detailed 
Qualifying and follow-up data for each pupil in a form in which 
any required fact could be seen at a glance. We therefore decided 
to make graphical profiles of the type shown in fig. 8 (p. 93) for 
each of the senior secondary pupils. As the technique is one which 
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we recommend to future investigators we shall deal with their 
construction in some detail. 

At the left-hand side we have the pupil’s scores in the seven 
Qualifying tests and examinations. The next point, marked Av, 
gives the unweighted average of the seven. Then follow three 
points giving the primary teacher’s estimates of the pupil’s practical 
ability (Tp), industry (Ind), and the Medical Officer’s grading 
for health (Н). 

On the right-hand side, the point between the first pair of 
vertical lines is the primary teacher's forecast of the pupil's success 
in the pos t-primary course actually selected. ‘The next three points, 
marked I, II and III, give the pupil’s performance in the first 
three years of the post-primary course; and the last point indicates 
the head master’s final estimate of success. 


NECESSITY FoR USING STANDARD SCORES 


Ordinary marks or actual scores would be unsuitable for the 
construction of such profiles. Even if all scores were in percentages 
they would not be comparable and would be misleading. If, for 
example, a pupil had уо per cent. in English and уо per cent. in 
Arithmetic, the graph would give the impression that he was 
equally good in both; but this would clearly not be the case if 
the mean score in English were 70 per cent. and that in Arithmetic 
60 per cent, If, therefore, we wish the scores to be comparable, 
We must relate them to their respective means. Such a procedure 
will enable us to use a fixed line of means running right across the 
graph, so that we can tell immediately whether the pupil is above 
ог below average in any particular measure. Even this method 
would be misleading if there were differences in the standard 
deviations, If a pupil is ro per cent. above the average іп 
English and in Arithmetic it does not follow that he is equally 
good in both; this depends on the standard deviations of the 
marks in the two subjects. 

For true comparability, therefore, we must take account both 
of the means and of the standard deviations, and this is done by 
the use of what are called standard scores. ‘These are obtained 
by finding the pupil’s deviation from the average score of his group 
and dividing it by the standard deviation of the group. If the 
Mean and standard deviation of the group are 60 per cent. and 
10 per cent. respectively, and if a pupil's score is 70 per cent., his 
standard score is thus +10. 
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The use of standard scores results in profiles in which all 
Scores and all differences are comparable, and there is th further 
advantage that every point has a definite significance in relation 
to a normal distribution. If a pupil is placed at +10, this means 
that, in a normally distributed group, only about 16 per cent. of 
the pupils would have a higher score; if at —10, only about 
16 per cent. of the pupils would have a lower score. If his 
placing is +20, only about 2:5 per cent. of the pupils would 
.have a higher score; and a standard score of +30 may be taken 
to indicate that the pupil is the best in his group. 

This, then, is the method which we adopted in making the pro- 
files. "The horizontal at the middle of the graph is the average line, 
and the dotted horizontals are drawn at intervals of то. In fig. 8 
we see at once that the pupil is of outstanding intelligence. He is, 
relatively, not so good in English; and we have a confidence in 


the reality and size of the difference that we could not have had 
if we had used ordinary marks. 


Èj 


CALCULATION or STANDARD SCORES FOR Success Marks 


There are many technical difficulties in the determination of 
standard scores for our present purpose, which can. best be 
illustrated by considering the „Ел; marks. We wish to deal with 
the group as a whole, and. the scores must therefore be strictly 
_ comparable for all pupils. 

The pupils are in five different senior secondary schools whose 
follow-up marks are not comparable. If we simply calculate a 
pupil's standard score from the mean and standard deviation of his 
own school, there is an error through assuming that the average 
standards of intelligence and attainment in all schools are the same. 
The standard „Ёл, scores have therefore to be corrected for 
differences in the average levels of the schools by arranging that 
they are given relatively to the mean and standard deviation of the 
complete group, that is, the pupils in all the schools if they were all 


marked on the same standard. All the Fay marks are then 
comparable among themselves, í 


We also wish the „Ех, marks to be comparable with those for 
iF ay and Fay; but the third-year group is more highly selected 
than the groups for the firs 


t two years on account of the dro ing 
out of weaker pupils. If, theref 4 PP 


У ] істезоте, a pupil has a standard score 
of -- Io in the third year he is better than one who has a score ОЁ 
- Ig in the first year. This difficulty can be overcome by making 
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the scores for all years relative to the first-year group. That is 

to say, when we give a transformed score of +10 for a third-year 

pupil this means that, if the whole first-year group had remained 

to the third year, he would have been +10 above the average of 

this group. Otherwise a pupil might appear to be falling back in 
1s secondary work, whereas he is actually improving. 

Starting, then, from the raw 3F 4y marks we have to make three 
transformations: (1) into standard scores, (2) to ensure compar- 
ability between school and school, and (3) to ensure comparability . 

etween year and year. 

The foundation of the second and third corrections is the 
scaling of the follow-up marks of the different schools on the 
comparable Qualifying examination marks. That is to say, the 
transformed follow-up marks for a school will have the same mean 
and standard deviation as the Qualifying examination marks for 
the pupils of that school. We can now treat the follow-up 
marks for all the schools as comparable, and use the mean and 
standard deviation of the whole group for fixing the standard 
Scores, It is, however, not necessary to scale the marks as a 
Separate step, Each of the three transformations is linear; there- 
fore we can make all of them at once by. means of a single equation” 
which, though more complicated, is still linear! 


А, Starting from the fundamental scaling equation (p. 44), and knowing the 
nature of the transformations to be made, there is no difficulty in arriving at the 
following formula for the complete transformation of a raw third-year score 
АЁ, for a pupil in School A into a comparable standard score АХ; : 


АҒУ 


АХ _Ma - Мо, " аз Afg, AME аба 
x со, бо, болб, 09, ACT, 
Where Mq, —mean Qualifying examination score of the whole Fay group 
for the five schools, 
Мо, 2 mean Qualifying examination score of the whole ¿Fay group 
for the five schools, 
аз = deviation of mean Qualifying examination mark of aFav group 
in School A from the mean Qualifying examination mark 
for the Ғау group in all five schools, 
AGr, standard deviation of зГлу marks for School A. 
The equation is linear, and when we put in the values of the constants it 
Teduces to 
AXg = — 3:631 +-068 АЕ, 
Thus, if a pupil’s raw Fay mark is 80 per cent., his comparable score 
18 +1.80. 
Different equations, simpler in form, are required for thé first and second 
Years; 15 equations in all are required. 
7 
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CarcuLATION or STANDARD Scores rog QUALIFYING 
Tests AND EXAMINATIONS 


The first seven data on the left-hand side of the profiles are based 
on uniform tests or examinations, and are comparable within the 
December group and within the June group. We knew, however, 
that the level of intelligence and attainment was higher for the 
June than for the December group, a difficulty which we sur- 


IQ - EQ -AQ 
3 


mounted by the method of scaling, using Р- asia 


bridge. 


Ртасікс THE ЁтуЕ-Ротхт Grapincs on THE PROFILES 


It was not necessary to place the gradings for industry, health, 
etc., with great accuracy, but it would have been misleading if we 
had placed, say, all the A gradings at the same o distance above 
the line of means. If, for example, 1 per cent. of the pupils jos 
graded A in health and 10 per cent. in industry, an А in hes 
would indicate a higher standard than an A in industry, and the 
À point for health should be higher above the line of means. | Ву 
a statistical procedure we can, from the distribution of the gradings, 
find the means and the ends of the ranges for the various categories, 


and these are indicated in the profiles by points with vertical lines 
through them.? 


* The following typical equation is that for converting Qe marks for the 
December group into comparable standard scores: 


Ferd оМое Т DOP 
Ор OPDOQe Ор Doge 
where Qe=raw mark, 
X —standard score, 
pMge=mean Qe mark for the December group, 
@=pMp - (0 43)Mp. 


* In these calculations we assume that the measures are normally distributed. 
We find the feet of the ordi i : 


of pupils graded in the various categories. Finally we calculate the means of 
the sections of the curves, which will, of с 


points of the ranges. For the primary teac 
grading, where the letters had a definite objective meaning, we did not use the 


actual percentages in the various categories; we simply divided the base-line 
of a normal distribution into five equal parts. 


References: K. J. Holzinger, Statistical Methods for Students in Education, 
р. 219. 


S. Dawson, An Introducti 


„Тит on to the Со; 
London: University of 
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CALCULATION OF THE AVERAGE OF THE SEVEN QUALIFYING > 
Tests AND EXAMINATIONS 


The average of the seven Qualifying tests and examinations is 
found by first adding the corresponding scores and then dividing 
by 7. Since each element in the numerator is in standard scores 
it might be thought that the averages obtained in this way would 
also be in standard scores; but this is not so. 'Their mean would 
certainly be zero, but the standard deviation would be less than 
unity by an amount which depends on the size of the inter-correla- 
tions. We had, therefore, to transform the averages into true 
standard scores by dividing by their standard deviation. 


STANDARD SCORES FOR QUALIFYING DATA NOT COMPARABLE 
WITH THOSE FOR SUCCESS 


There is one sense in which the data on the profiles are not 
comparable, A pupil may be above +10 in the Qualifying data, 
but well below the average line in the success data. At first sight 
it would appear that he has fallen off very badly. It must be 
remembered, however, that his scores on the left-hand side show 
that he is above the average of the whole Qualifying group; while on 
the right-hand side his scores are below the average of the highly 
selected group that enters the senior secondary courses. 

To avoid confusion we shall now denote the standard deviation 
of Qualifying scores for the complete Qualifying group by sigma, 
using c for the standard deviation of the post-primary group with 
which we are dealing. 


Tue Ѕновт HORIZONTAL Lines ON THE PROFILES 


There are two short horizontal lines on the profiles. The one 
at the left-hand side is drawn horizontally at ++7 sigma, and may 
be regarded as an entrance pass-mark for the senior secondary 
courses. Pupils whose average scores lie above it have more than 
а 50 per cent. chance of success. Its determination will be 
explained later. If a pupil’s success scores lie above the short 
horizontal at the right, it may be taken that he has made good in 
the secondary course. 

SAMPLE PROFILES 


As these profiles have more human interest than the Tables of 
correlations with which we have hitherto wearied the non-mathe- 
matical reader, we shall now give a few samples which will afford 
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: i rd 
practice in their interpretation. We shall deal first with ie i 
cases, where the pupil's performance in the secondary course i 
conformity with his Qualifying data. 


THE PUPIL WITH THE HIGHEST QUALIFYING SCORE 


Fig. 8 is the Profile of a pupil who has fully borne ph 
early promise; he set us an easy problem at the Qualifying re 
and our answer would have been correct. This is a boy оп w. we 
fortune has smiled. He comes from a well-to-do home, г ары 
imposing list of good qualities, “perseverance, originality, ded him: 
and self-reliance,? and his primary teacher rightly describe ae 
as a boy of ‘great possibilities” “The fact that Music, че. һе 
Bible are the only subjects of the secondary course whic ег 
regards as difficult is, we Suspect, an indication of his tastes ra 


A : ith 
than of his abilities. He obtained his Leaving Certificate wit 
great credit in 1941, 


THE PUPIL WITH THE LOWEST QUALIFYING SCORE 


For comparison we give, in fig. 9, the profile of the pupil bos 
had the lowest average score in the whole Qualifying gen is 
follow-up scores are relative to the average of the backward c 1 d 
to which he was sent, The two form a striking contrast and са 
be of interest to апу who still believe that all children are bo 


equal in Capacity, or that they should all have the same posts 
primary education, 


^ CLEAR ADMIT 

Те profile in fig. 10 is that o 
Without hesitation, His standa 
+-50 are just what o. 


f a pupil whom we should admit 
rd follow-up scores of 4-48 to 
ne would expect from his Qualifying tests. 


^ BORDER-LINE ADMIT 
The next profile (fig. 11) is that of a 
English but weak in Arithmetic, She is just above the border- 


line, and our hesitation as to her fitness for АД ыы е used 
by the low health grading. We find her салаша 


сә; > aS we should expect, 
just above the success line in the Secondary course, 


girl of good ability in 
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Fic. 8.—The pupil with the highest Qualifying score. 


27 


m 
IQ Ge E Тез ба А Tos Av Tp Ind. н Fc. I I M 


Fic. 9.—The pupil with the lowest Qualifying score. 
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Fic. 11.—A border-line admit. 


PROFILES. 95 


A CLEAR REJECT 


The next (fig. 12) is а case where we should reject without 
hesitation. This girl is clearly below the border-line in intelli- 
gence, English and Arithmetic, and her primary teacher perhaps 
erred on the side of generosity in giving her a forecast of С. In 
the first two years of the secondary course she clung precariously 
to her hopeless position of about —10, plunged downward in the 
third, and left school to enter a business college. 


IQ Qe E Tes Qo А Tas Av. Tp Ind. Н Fc. I I II S 


Fic. 12.—А clear reject. 


A HOPELESS REJECT 


If the parents of this boy had seen his profile (fig. 13) one 
Wonders whether they would have persisted in sending him to a 
senior secondary school against his own desire. Qualifying at 
13 years 9 months, he is weak in everything, has an E grading in 
industry, and only one marked personal quality, namely, laziness. 
His Р later performance amply justifies the primary teacher's 
Pessimistic forecast. The secondary report states that the boy 
was fit only for a backward class; he had no interest in school- 
work, and finally started truancy in order to induce his parents 
to withdraw him. 
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Fic. 13.—A hopeless reject. 


These graphs tell six human stories of children very differently 
endowed. They range from one pupil who, we trust, will be a 
leader in some department of culture, industry or commerce, to à 
number who seem destined to be merely hewers of wood and 
drawers of water. And while they all have their individual 
educational claims, these were certainly not met by a system under 
which all but one were allowed to enter a senior secondary school. 
The admission of pupils with no chance of success is harmful to 
the pupils themselves, and imposes a heavy burden upon the 
senior secondary teachers; but there is a third aspect which is 
sometimes overlooked. In these democratic days we think so 
much of the claims of the average and dull that we run the risk of 
forgetting those of the intellectual élite who will be the future 
torchbearers in cultural, social and industrial advance; and it is 


partly in the interests of this élite that improvement in our methods 
of selection and guidance is necessary. 


DISCREPANCIES 


It must not be thought that the system is infallible. Later we 
shall reduce the discrepancies to proper Statistical form, but mean- 
? 
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time we present two cases where the reader, having had his lesson . 
in forecasting, can make his predictions and find himself wrong. 
The two pupils in fig. 14 are at very nearly the same standard in 
the Qualifying data, and both qualified at 12 years of age. If the 
reader has profited by his lesson, he will have no hesitation in 
admitting both pupils, confident that they could be clear, but 
undistinguished, successes in a senior secondary course; if he has 


+3 


+2 


+1 
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Fic. 14.— Discrepancies between Qualifying score and later success. 


Blven some care to his preparaton he will predict, not only that 
they would be Successes, but that in their success marks they should 
oth have a score of about +530. 

r The right-hand side shows how far he is in error; and his satisfac- 
tion that at least one of the pupils is a success is tempered by the 
fact that this pupil is far more successful than she ought to be. 

ST success score, instead of being 4-30, is уе] оуег+1:5о. In 
her Qualifying reports she is described as self-reliant and specially 
talented in Art. In the secondary course she took two foreign 
anguages, French and German, and the head teacher’s explanation 
ОЁ her rather remarkable success is that the mother-had to bring 
Up the family without help. 'The pupil was extremely eager to 

4 
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take full advantage of educational opportunities which, she realised, 
had been obtained for her with difficulty and sacrifice. К 
Тһе boy is an entirely different type, yet there was nothing 
in all our Qualifying records that could have saved us from the 
mistake of admitting him. Не was a clear failure from the start 
of the secondary course, is reported to have been lazy and in- 
attentive throughout, and left after two years to become a salesman. 


TWINS 


'The two following examples (fig. 15) do more credit d 
system. "They afford a picture of twin boys who show striking 
resemblances. Both have exactly the same IQ, the same mar S 


iQ Ge Е Tes Qa A Tas Av. Tp Ind. H 


Fic. 15.—Twins. 


special talents and the same personal qualities, Prediction У/45 
Т? here, A po correct. They are, however, both es 

etter in the secondary course than w. 1. pected, 
even with their high Qualifying a SN CRUCIS 


a С 2 ing average of over +2 sigma. Both 
obtained their Leaving Certificates with great credit a 1941. 
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HEALTH - 


Fig. 16 presents the data for a case of no little difficulty, and 
included to emphasise the point that factors other than examina- 
Чоп and test results should be considered in selecting and guiding 
Pupils. It is that of a girl of a fine type, with everything in her 
favour except health. She is a perfectly clear admit, highly 
intelligent, interested in cultural pursuits, persevering and in- 
dustrious; a girl of originality and initiative, with special musical 


is 


Fic. 16.—An able pupil with a low health grading. 


gifts and from a good home. But the Medical Officer’s health 
grading is D, and his confidential report shows that she has heart 

1ѕеаѕе and anemia. The report from the secondary school 
mentions that she is ‘delicate and often absent.’ Despite her 
physical handicap she has done well in her secondary course, 
though with distinct signs of falling off. ` 

It is doubtful whether a girl of this type should be encouraged 
to face the strain of a full Leaving Certificate course, with the 
temptation to venture on the still more intense strain of an honours 

gree course at a university. 
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STEADINESS AND VARIABILITY 


Fig. 17 raises a point which we have not yet had time to 
explorefully. Some pupils are remarkably steady in their peron 
ance, others are extremely variable; and it is not always the steady 


+||-----------$------ 
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Fic. 17.—Steadiness and variability. 


pupil who is the good one and the variable pupil who is the ps 
one. The variable pupil, however, presents us with the mo” 
difficult problem, particularly if he is near the border-line. 


SPECIALISED ABILITY 


As a rule the pupil who is good in English is good in Arithmetic 
but occasionally we come upon instances like the following: T 18 
profile (fig. 18) is that of a girl with aspirations to become à music 
teacher, who has high literary gifts, but is very weak in Arithmetic- 
This is no doubt partly responsible for her failure in the secondary 


course, where she found special diffi i 1 8 
we shall see later, we should have und er. 


ане ever алар bel EQ cs a this pupil for her wea 


EE 7 considerably above the 
admission line on the average. This, of cou 156; P ns merely 


that in the existing secondary courses she would be unlikely to be 
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қ КЕШЕГІ 3.3.0 1—1 
IQ Qe E Tes Qa А Tas Av Tp Іпа. Н Fc. I п aus 


Fic. 19.—A pupil who has made a flop in one subject. 
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a success. Whether, in a sound educational system, an aspiring 
musician, with literary gifts of the order of +1-4 sigma, should 
be debarred from the chance of attaining her desire because of 
weakness in Mathematics, is open to question. 


A FLOP 


Our last illustration (fig. 19) at this point is that of a boy who 
has made a really thorough flop in his Arithmetic examination. 
With him no difficulty would arise, for he is a sure admit; but a 
collapse of this kind in a border-line case might make all the 
difference between admission and rejection if we had no means 
of conyincing ourselves that the low mark was due to accident. 
He is not doing quite so well in his secondary course as we should 
expect, due, according to his head master, to laziness. 


i 
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CHAPTER XIII 


USE OF PROFILES TO DETERMINE THE NUMBER 
OF MISFITS BY DIFFERENT METHODS OF 
SELECTION 


GENERAL PROCEDURE 


We know which pupils have turned out to be successful in the 
secondary course, and which have proved to be failures. If we 
take the standard average scores on the profiles as the basis of 
selection, and use a pass-mark of, say, +1 sigma, we can now 
quickly deal the profile cards into two bundles, those whom we 
should admit and those whom we should reject. These two 
bundles represent the answers which, on this selection system, we 
should give to the 452 problems set to us by the applicants for 
admission to the senior secondary schools. But we now know the 
Correct answers, and we can count our mistakes. 

These will fall into two categories: those whom we should have 
admitted but who have failed (admit-fails), and those whom we 
should have rejected but who have made good (reject-successes). 
These two categories together are the misfits. 

With this technique the possibilities of rapid experiment are 
almost limitless. Having ascertained the number of misfits with 
a pass-mark of +1 sigma, we can proceed to make a similar count 
for the same tests but with a pass-mark of 4:5 sigma. We сап 
also deal out the cards on the basis of, say, IQ alone, and find 
how the number of misfits compares with that for the complete 
battery. 

To the non-mathematical reader this very direct attack on the 
problem will doubtless carry greater conviction than the correla- 
tion results. It will also assist him in his interpretation of correla- 
tion coefficients. The statement that ror —:770 and тур =:691 
may convey little more than a vague idea that the examination 
isa better basis of prediction than the intelligence test, leaving 
him in doubt on two important points: 
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ivi ne. 
(1) Whether the difference is an inpor ora ides e will 
(2) Whether a correlation of `770 means t nx р 
be virtually perfect, indifferent ог good. u-—- 
He will find it more intelligible if we can tell him - Dudes 
of mistakes which would result from the use O 
examination marks and of intelligence quotient. 


SENIOR SECONDARY SCHOOLS 


N THE 
FINAL АѕѕеѕѕмЕМТ ОЕ Success AND FAILURE D 
бесохракү COURSE 


ecision 
The use of the above method clearly demands a Бие E in 
on the question whether the pupil has or has б. : following way. 
the secondary course; and this was reached in the ach case was 
` At the end of the third year of the follow-up hich there was 
scrutinised separately, and we set aside those in n c who had à 
-по possible doubt. The remainder, including vcre discussed in- 
Success grading of C on the follow-up cards, Were of the hea 
dividually with the head teachers. If, in the opinion ЖӨКЕ с 
teacher, the pupil had more than a 50 per cent. chance m aix уеегі 
a minimum Leaving Certificate in a suitable course i c he was 
he was counted a Success; if less than a бо per cent. chana EE 
counted a failure. Ifa pupil was unlikely to make goo unted à 
such causes as ill-health or bad home conditions, he was he ability 
failure even though the head teacher felt that he had the 
to make good under better conditions. | А ched it, 
Тһе standard is not high, but we felt that, if a pupil тей from а 
no Education Committee could honourably exclude him ж 
senior secondary school. 'The qualification *in a suitable cou 
с hea 
was intended to meet a not uncommon case where the ш 
master was of the opinion that the pupil was in the wrong Fou 
И that n would have made good if he, or his parents, had ma 
a better choice. 


ok: 


n until he had discus 
with various members of his staff; For gy, 


Frequently the 
sed the problem 
pupils he was usually 
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good enough to send a full written report giving detailed reasons 
for his decision. 

One point that emerged clearly from these discussions was that, 
Whatever variation there may be in the standards of marking of 
the individual schools, there is almost perfect agreement among 
the senior secondary head teachers as to the border-line which 
Separates those who will gain the Leaving Certificate from those 
who will not. With the comparable follow-up scores before us 
we found that, after the interview with the first head teacher, 
we could tell almost exactly when the others would begin to 
hesitate, when they would sail into the smooth water of the sure 
fails, and where they would finally draw the line. We owe a deep 
debt of gratitude to the secondary head teachers for the courtesy 


and patient forbearance with which they gave us their help in this 
difficult task. 


STANDARD FOLLOW-UP Score ron THE BorDER-LINE 
BETWEEN Success AND FAILURE 


When all the decisions had been made we determined, by a 
method which will be explained later, the point on the scale of 
standard follow-up scores separating the successes from the 
failures. It was at —:51с for the whole group and at almost 
exactly the same point for each separate school. 


ÅPPLICATION оғ тнк Metuop or MISFITS TO TEST THE VALUE 
OF THE Quatiryinc EXAMINATION IN SELECTION FOR 
Apmission To SENIOR SECONDARY SCHOOLS 


Everything is now in readiness for the application of our post- 
mortem procedure; and we proceed to show how many mistakes 
we should have made if we had used the Qualifying examination 
45 our test for admission. 

Fig. 20 is a picture whose story should be of considerable interest 
to all who have the responsibility for selection and guidance at the 
Qualifying stage, and the non-mathematician will find it more 
illuminating than the bare statement that ғор = "770. 

There is no horizontal scale; so the diagram, while showing 
Whether the pupil was good or bad on the entrance test, does not 
Show the degree of success which he attained in the secondary 
course—merely whether he was, or was not, a failure. The 
vertical scale is that of the standard Qualifying examination scores. 
The pass-mark of the examination is taken as +60 sigma. 

4* 


o6 


- 
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ADMIT- SUCCESS ADMIT- FA 


5209 б 


Е IN QUALIFYING EXAMINATION 


ээй ® 4% 


ШІЗІКТЗІ 918294 
Фофаю HO 


PASS MARK 


STANDARD SCOR 


REJECT- SUCCESS 


hiv 
FELIS 


Fic. 20.— Mistakes which woul 


m ad 
ld have been made if the Qualifying examination h 
been used as еп 


trance test for senior secondary courses. 
We now count our misfits with the following results: 


Successes 


Failures Total 
Admit . 283 38 321 
Reject . = E 21 74 95 
Total . 304 112 4161 
2 [n order to ensure comparability of the гези! 


lts which follow it was necessary 
to omit pupils who had absences from any of the sever tests, 
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Out of the 416 applicants we should have admitted 321, and of - 
these, 38 would not have made good. We should have rejected 
95 pupils, 21 of whom would have been wrongly debarred from 
the opportunity of taking a senior secondary education. The 
pa of mistakes we should have made is 59 out of the total 
of 416. ; 


Rance or ENTRANCE ATTAINMENTS OF THE MisriTs 


One of the most interesting points brought to light by the 
diagram is the scatter of the misfits on the entrance attainment 
Scale. Pupils with a Qualifying score as high as +1:50 may be 
failures; pupils as low as —-10 may make good. The band of 
uncertain prediction lies between the lines CD and ET. 

Some have aspired to a selection system where no pupils would 
be admitted to a senior secondary course who would not make 
good, and are disposed to be critical of the presence of any failures. 
Тһе diagram shows that it would be possible to satisfy their 
demands. We should fix the pass-mark at CD; but the purity 
of the secondary classes would be purchased at the cost of denying 
a secondary education to no less than 223 pupils fit to profit 
by it. Others, taking a more humane view, desire to reject no 
pupils who would ultimately make good. Their demands could be 
satisfied by lowering the pass-mark to EF, but at the cost of 
admitting 95 pupils who would clog the educational machine and 
suffer the depressing effects of early failure. : 


Tue Pass-Mank 


This leads to the important problem of the correct fixing of 
the pass-mark, and we use the diagram to introduce the reader 
to the method by which we studied it. With a pass-mark AB 
we have 59 misfits. Suppose we raise it to +:70, we reduce 
the admit-fails by 4, but we increase the reject-successes by 8, 
giving a net increase of 4 misfits. On tabulating the results of 
raising or lowering the pass-mark in this way we come upon ап 
important law. 

The law, which we shall find, to be general, is that for any 
entrance test there is a pass-mark at which the number of misfits 
18 a minimum—clearly a law of fundamental importance. 

One other notable fact that emerges from a study of Table 
XXXVII is that, at the point where the misfits are a minimum, 
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TABLE XXXVII 


isfits 
Effect of changing the pass-mark on the actual number of m 
(Entrance test, Q) 


Jlisfits 
Pass-mark Admit-fail Reject-success Mis 
223 
-- 1-60 sigma o 225 
trgo o 4 186 139 
+1520 9 130 99 
+1-00 18 81 70 
+ -80 27 43 63 
+ -70 34 29 59 
+ “бо 38 e 65 
+ -50 51 IT 70 
+ 40 бо 10 75 
+ +20 69 6 89 
*00 86 3 95 
— «20 95 o 


т 
rae , e numbe 
the number of admit-fails is considerably greater than th 

of reject-successes, 


Tue Correct Pass-Mank E 
Я 6 s estion 
It will now be convenient to consider the general qu 
the position of the pass-line. 
can throw some light, but 


Scientific facts. Our finding 
we think it does 


A int. 
As we know of no authoritative Pronouncement on the point, 
we shall assume that the two types 


of mistake are equally йш, 
That being so, we Suggest that the pass-mark is drawn correctly 
when the number of misfits is at the minimum, А 
We shall find later that the minimum always comes at a definite 
point, at which: 
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The pupil who is admitted has more than a 50 per cent. chance 
of succeeding; the pupil who is rejected has less than a 
50 per cent. chance of succeeding. 

In all cases likely to arise in Scotland, the number of admit- 
fails will be greater than the number of reject-successes. 


If we err, therefore, it is on the side of giving the child his 
educational chance. 

Another possible pass-mark is the point at which the number of 
admit-fails is equal to the number of reject-successes. This has 
à specious air of fairness, but a glance at Table XXXVII will show 
that it would give a larger number of misfits, and its unsoundness 
Will be more clearly disclosed later. 

We shall in future always refer to the point at which the number 
of misfits is a minimum and the child has just an even chance of 
Success as the correct pass-mark. 

€ now proceed to compare the predictive value of the various 
tests, examinations and batteries by finding the number of misfits 
each would give if it were used with its own correct pass-mark, 


Meruop or TABULATION 


Tables of the following form were prepared for all Broups, 


senior and junior secondary, backward and retained. 
І 


Day Shorter 


Class i oss School post- Backward 
interval eia caving — Certificate primary class 
Certificate Certificate (Lower) сое 
5* ЕР S; UE 5 Е 5. 8 SF 
+40-49 4 17 7 1 21 о 21 o o o 
gms. (REO ІУ ii (Lg "IN 
* S=Success T F=Failure 


In the case of the Junior Leaving Certificate a pupil was graded 
à success if, in the opinion of the head teacher, he had more than 
а 50 per cent. chance of obtaining the Certificate in three years. 
А similar test was applied for the Day School Certificate (Lower), 
the time allowed being two years. j 
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SFITS 
DETERMINATION OF THE Міхімум NUMBER OF М: 
FOR А 'Trsr ок BATTERY 


TT т 
There are two difficulties in determining the minimum nume 
of misfits. In the first place, the minimum may lie wit елу 
interval; in the second place, the numbers within аы of an 
of -1 sigma are small, and there is consequently а od : » 
steadiness due to the chance fluctuations of small num lect of 
must therefore try to introduce some of the steadying 
the numbers above and below the pass-line. ] number 
If we take Table XXXVII as our illustration, the wes д fairer 
of misfits is 59 with a pass-mark of +-60 sigma. 'To n Jot the 
estimate of the general tendency of the Table we cou is an 
number of misfits against the pass-marks, smooth the env 
find where the smooth curve marks the minimum. sno curtesi 
Generally speaking, there are two ways of smoothing cs. AD! 
mathematical methods, like the method of least squat volve 
graphical methods. The first method is the soundest but ried out 
very heavy calculations. The second method can be car Ду wit 
rapidly without calculation, and works very carp n we 
certain types of curve. The essence of such methods is fitan 
replace each point on the curve by the weighted average 9 
of the points above and below. +. vends t9 
The disadvantage of the graphical method is that it t€ со 
make a concave curve a little less concave, and all the pint 
misfits are of this type. To avoid this we tried various exci 
but finally decided to plot separately the curves for admi ere 
and reject-successes. These, as will be seen from fig. 21» 
much more suitable for graphical smoothing. diness 
The figure shows how the smoothing removes the unste? шт 
which comes from small numbers. 'The number of misfits 


any pass-mark is found by reading from the graphs the numbers 
of admit-fails and reject-successes, and adding. 


DETERMINATION Or THE Correcr РАѕѕ-МАКК 


The correct pass-mark was determined in two ways. At the 
correct pass-mark the number of misfits is a minimum; therefore 

1 If we have a series of points у,, у,, Jh 
corresponding to y; a ruler is 


2. «Jy to find the smoothed point 
vertical distance between y, 


placed through Уз апа y, The mid-point of the 
and the ruler gives the smoothed point. In this 
way we replace уз by Эһ%2Уа уу 


In refractory cases the process can be 
repeated with the smoothed points. 


NUMBER 


PERCENTAGE OF FAILURES 


ADMIT- FAIL 
REJECT - SUCCESS 


PASS-MARK (SIGMA) 
Fie. 21.—Smoothing of curves of admit-fails and reject-successes 
(Battery ТО+5+ Ts). 


— 
-:005 *:395 %:795 +1195 *1:595 


PASS-MARK (SIGMA) 


Fig. 22.— Determination of correct pass-mark from percentage of failures 
(Battery IQ 4- S4- T3). 
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we can find it from the above-mentioned graphs. "Тһе pupil n 
an even chance of success; and the correct pass-mark will es ГЕ 
point at which the 50° рег cent. line cuts the curve i-e 
plotting the percentage of failures for each class interval. 1 
is shown in fig. 22. E 

Figs. 21 and 22 are both in respect of the battery IQ +5 + Ts, 


and by both methods the correct pass-mark comes out to be 
673 sigma.1 


Correcr Pass-Manks ror Varrous Tests anp BATTERIES 


The pass-marks obtained in these two ways for all tests ang 
batteries are given in Table XXXVIII. The batteries are arrange 


5 t Á x P ; st 
in order of the correlation coefficients, which are given in the fir 
column. 


TABLE XXXVIII 


Correct pass-marks in terms of sigma for all tests and batteries 


Pass- Pass- 
Corre- | mark mark Average IEEE à Theoretical 
Battery lation | (so per (mini- d pass- P pass-mark 
with F| cent. mum mark oot cd 
chance) misfits) ML. 

о1о [оо l o w o 

— SR n ө | uu 
10-0-7% .| с, “696 68 
IQrQ ST. воо | 6 | 77 | or ES 7o ЗА 
e +S+Ts . 799 701 701 "600 s 69 66 
ue 42041 -786 676 644 | +032 | .6бо 69 66 
ae з $ 783 680 713 | —-o33 | -696 +68 562 
+Q+ -783 -654 -676 —-022 | -66 -68 66 
UE +Ts .| 282 -673 +673 “осо on 68 66 
Meath + «| 7779 | -680 46 | +.034| .663 67 65 
d FS" 274 ate 649 | +-orr -654 -66 -65 

Е -631 -6 E Ы 
S +Ts. -| +764 -666 p 55 p ip Fs 
IQ*S .  . 6 | lm Soe o 6 FR n 
Ts . -| +720 +642, +566 iso 229 соз . 
5 АНТЫ Aes езе |i wees 8 Pe E 2 
Q E а 475 475 +000 Е 3 

j 475 -57 55 


The discrepancies between 
the two method: i h 
? 3 s of finding the 
pass 85 Es in column (4), аге Not great, and ан 
sigma only for ТО +5 and for Ts То get a more reliable estimate 
1 In determining the correct pass-mark i; 


roS t hi k, ; ; 
standard deviation of IQ +Q + Ts is less dia Шу be kept in mind that the 


USE OF PROFILES TO DETERMINE MISFITS 113 


of the pass-mark the two measures were averaged, giving the 
results in column (5). 


THE HIGHER THE FOLLOW-UP CORRELATION FOR THE BATTERY 
THE HIGHER THE CORRECT PASS-MARK 


Column (5) reveals the important law that the higher the 
predictive value of the battery the higher should be its pass-mark. 
In the general discussion, to which this chapter is ап empirical 


SMOOTHED ACTUAL CURVE 
THEORETICAL CURVE 


PASS-MARK (SIGMA) 


70 21 2:72 078 274 275 76 277 78 :79 80 


CORRELATION 


Fic. 23.—Fall of pass-mark with fall of correlation: theoretical curve 
and.smoothed actual curve. 


preamble, the reason for this will be given, and we shall find that 
the theoretical curve of fall of pass-mark with fall of correlation 
is smooth. We may therefore assume that we should get more 
reliable estimates of the pass-marks by smoothing the actual curve. 
The result of the process is shown in fig. 23. 

No special significance attaches to the fact that the theoretical 
curve is a little lower than the actual one: this is due to the skew- 
ness of the distribution of the senior secondary group. It is 
required at the moment merely to act as a check on the general 
sweep of the actual curve; and when we consider the size of the 
vertical scale it is clear that the agreement is satisfactory. We 
therefore take the points obtained from this smoothed curve as 
the best estimates we can get of the pass-marks for the batteries; 
and these are given in column (6) of Table XXXVIII. The pass- 
marks obtained from the theoretical curve are given in column (7). 
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For assessing the significance of the small possible errors 1n the 
determination of the pass-marks we give the following guide: 
An error of -03 sigma in fixing the pass-mark would increase 
the misfits by about 1 per cent. i boat 
An error of -o2 sigma would increase the misfits by abou 
I in 200, 


= 5 : about 
An error of от sigma would increase the misfits by abo 
I in 500. 


я ERIES 
Момвев or Misrits ror tue Various Trsrs AND BATTERI 


We now come to the misfits for the various batteries, full € 
of which are given in Table XXXIX. For the sake of those w in 
regard all statistical manipulation with suspicion we pe ы 
column (2), the actual minimum misfits obtained from the 95 
tributions with a class interval of -1 sigma. "These, however; ү 
not the exact minima, which would usually fall within a clas 
interval. In columns (3), (4) and (5) we give the minima obtaine 
as explained above from the smoothed curves. 


TABLE XXXIX 
Number of misfits for the various tests and batteries 


Corre- 

Actual minimum misfits | Minimum misfits for ино 

Corre- | Actual (smoothed) normal correlation corre 

lation | mini- үл” 
i 1 Р 

Battery with | mum i T 
F | misfits | Admit- | Reject- | ar; py, | Admit- | Reject- | ұры | 46 

- fail | success | Misfits Tl jxusceis Mis misfits 

(col. (5)) 


omrol ol ||| | o] o 


IQ+Q+Ts .| -804 56 38-1 18. .8о2 

е 4 6- т 24- 68-4 
чыч Иа 552 б. | апу | 62 | 22 
Q+S+Ts . -790 56 


374 20:0 s 6:0 24-8 o8 793 

Quacer| zi 55 1385 | оо | 2% 521% 716 | 792 

to+ors ^| 75 | 3 233 дәл | 32 | 469 | 24:9 | 718 T 
S 9 "I 6. К I - 

oon d а ROM 2) 32.5 ES аға 782 

GF eon Ыл $ | 45h | 72 | овы | azs | ао | 725 p 

3 5 |58922 156: 8.1 : FEE 

9. 2 + 779 59 41-1 20-4 бз duo 227 Po "убо 

Н +| +764 56 40-2 20.1 60-3 49:6 aci a 1770 

o * | +736 62 399 | 251 64-1 53-7 22% ar 736 

8 с; > 2729 59 48-0 13-0 бізі 558 254. 552 763 

55: : ШК ei 426 | 27-4 | 68:0 586 | 25. 84- 
IQ . +| -691 69 471 | 222 54 4o 7 


69:3 | 596 


25:3 84-9 +69° А 


— eA 
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There is no serious discrepancy between the actual minima and 


- those obtained from the smoothed curves except perhaps іп the 


case of S+Ts, where a study of the curves leaves no doubt 
as to the greater soundness of the smoothed result. The way 
in which these deviations arise may be explained by reference 
to fig. 21. There the minimum is indicated by a vertical line 
near which the actual point on the admit-fail curve lies below 
the smooth curve. For the reject-success curve the opposite 
holds. Here, therefore, the actual minimum would be close to 
the smoothed minimum; but if the actual points had fallen 
below the smoothed curves in both cases the discrepancies would 
have been additive, and the difference considerable. 

The numbers of misfits range from 56-5 for the best battery to 
69:5 for the poorest. These should be considered in relation to 
112, which is the number of misfits we should have had if we 
had dispensed with the entrance examination and admitted all the 
candidates. Our poorest battery gives a correlation with F of 
“бол. By methods described in Chapter XIV we can estimate the 
numbers of misfits for batteries giving lower correlations. These 
are as follows: 


Correlation with F Minimum number of misfits 
6 82 
ur 94 
+ 105 
3 III 
2 112 
“1 112 
оо 112 


From these it will be noted that entrance examinations giving a 
correlation with F of less than -4 are of very little value. 


ORDERS or PREDICTIVE VALUE BASED ON CORRELATIONS 
AND ON MisriTS 


_ The relationship between the two orders will be shown later 
in graphical form. Meantime, an inspection of columns (1) and 
(5) of Table XXXIX will show that the conformity between the 
two is fairly close. ‘The first six batteries are in the same order 
on both bases; and the differences which cause the later dis- 
crepancies in order are small. 

It is, of course, not necessary that the orders should be the 
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same. "Тһе correlations are based on the whole of the data, 
and give the order of discriminative capacity of the batteries 
throughout the scale: the misfits give the order of discrimination 
at the border-line. As far as practical application is concerned 
we might put the position in this way. The order of misfits 
is the more significant for the Education Committee which is 


THEORETICAL CURVE 
=-= ACTUAL CURVE (2000660 
ACTUAL CURVE 


NUMBER OF MISFITS 


CORRELATION 
Fic. 24. i 
G. 24. Comparison 9f actual and theoretical number of misfits 
or different sizes of correlation. 


the selection of 
the order of 
ry head master 


pupils for admission to 
correlations is the more 
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Тһе fluctuations are somewhat less for the smoothed actual 
curve than for the actual curve; but they are far from completely 
eliminated, and give the impression that the deviations are real 
and not chance variations. The point A corresponds to О, and 
B to Ts; and while Q has the higher correlation, Ts gives the 
fewer misfits. It may well be that this is no mere accident. 
Q may get its high correlation by properly placing the better 
Pupil, while the teachers’ estimates may be particularly dis- 
criminating at the border-line. 


Comparison BETWEEN ACTUAL AND THEORETICAL 
Момвевв or Misrits 


The theoretical and the smoothed actual curves differ in two 
Ways. In the first place, the actual numbers are all much less 
than the theoretical. This is no doubt due, in small part, to the 
under-estimation of the correlations through a cumulation of 
small losses, but mainly to the fact that the distribution of the 
senior secondary group is tailed. In the second place, the actual 
curve has a slightly smaller slope than the theoretical. This is 
related to the first difference, as can be seen if we imagine the 
curves completed to the ends, 70 and r=1. 


CORRELATIONS WHICH WOULD GIVE THE ACTUAL 
Момвевѕ or Misrits 


Most of our data are in the form of correlations, and ultimately 
we shall gather together all our evidence in order to get a final 
order of predictive value for the various tests and batteries. It 
would therefore be convenient if we could put the order based 
on misfits in this comparable form. Since the theoretical and 
the actual numbers of misfits are not the same, we cannot do so 
by finding the normal correlations that would give the number 
of misfits for each battery. The only method is to draw a smooth 
curve through the points for the actual smoothed curve and read 
the correlations from it. In so doing we are not assuming, 
contrary to what has been said above, that the actual smoothed 
curve should have the same smooth sweep as the theoretical one. 
We retain the order of the misfits and simply put the data into 
another form comparable with correlations. "These fictitious 
correlations are given in column (9) of Table XXXIX. 
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Composition OF Misrits 
From Table XXXIX it will be seen that there are considerable 


fluctuations in the Proportions of admit-fails and reject-successes. 
This is due to the fact that while the total number of misfits is 
Dot very sensitive to small changes in the placing of the — 
line, the proportions are, "The numbers in the Table shou 


therefore be taken as Біуіпр merely а rough indication of the 
proportions. 


JUNIOR SECONDARY SCHOOLS 


DISTRIBUTION OF THE Juxior Seconpary Group 


The fact that the distribution of the senior secondary pupils 
is tailed caused certain differences between the theoretical and 
the actual results, As will be seen from fig. 57,1 the distribution 
of the junior secondary group is more nearly normal and we 
should anticipate that the correspondence would be closer. 


Quauiryine Pass-Mark FOR Success IN THE 
Junior Leavine Севтігіслте 


As for the senior secondary courses, we determined the point on 
the standard scores for IO 4 Q-S 4 Ts at which the pupil has a 
50 per cent. chance of Success in the Junior Leaving Certificate. 
For this battery the correct Qualifying pass-mark is as follows: 


Pass-mark Pass-mark "Theoretical 
(50 per cent. chance) (minimum misfits) pass-mark 
7:595 sigma +595 sigma --:597 sigma 


The border-line for Success in the 
about -1 sigma lower than that for th 
standard for the Certificate is therefor, 


Junior Certificate is only 
е Senior Certificate. The 
е fairly high. 


Table XXXVIII. 
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Момвек anp Composition or Misrrrs ІМ SELECTION OF 
Рири$ ror Junior Lravixc CERTIFICATE 


Using the methods already described, we can find the number 
of mistakes we should make if we used the battery IQ +Q +5 + Ts, 
with its correct pass-mark, to select pupils who should take a 
Junior Leaving Certificate course in junior secondary schools. 
The total number in our junior secondary group was 1,975. 


Admit-fails Reject-successes Misfits 


Actual = 4 164 258 аре 
"Theoretical . А 143 276 119, 


CHAPTER XIV 
GENERAL PRINCIPLES OF SELECTION TECHNIQUE 


Wz have, by empirical procedure, come upon several laws ee 
ing the operation of the selection process. Since these a 
be specific to our data, the time is ripe to gather the к 
points together in a theoretical treatment which will show 

far they are general. n 

п аа Chapter is an attempt to systematise а ашар 
of principles which we encountered from time to time in wor We 
up the Inquiry data. Starting from peculiarities of our --— 
we made various explorations into the field of pera ч | 
found that а surprisingly large number of laws could be de pee 5 
mathematically without experiment. We felt that our pes 
picture of the selection situation was clarified in this way, and 1 
hope that this chapter may do the same for our readers. eap у 
it brings out principles which should be in the minds o Es 
responsible for selecting pupils for secondary education О 
awarding bursaries or free places. 

Our honesty in recommending the non-mathematical reader to 
omit Chapter VI may give him greater confidence in our advice that 
he should read this опе. He should not be deterred by the fact 
that, at places, it has a mathematical appearance; this is de 
А certain amount of difficult mathematics undoubtedly is involve 
in some of the Operations; but we believe that, by taking the 
results of these operations for granted, he will be able to follow 


the drift of the reasoning and understand how the various 
principles are reached, 


Tue Scatrer Dracram 
Fig. 201 shows whether the pupil is a high or a low admit, 
but gives no indication whether he Was a distinguished or merely 
a border-line success in the secondary school. We can display 


the results in a way that will give us both pieces of information 
by a figure of the following type: 


1 P. 106. 
120 
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Success in secondary school 


Fic. 25.—Scatter diagram showing position of pupil both in entrance test. 
and in success. 


The pass-mark on the entrance test will be represented by a 
horizontal line like AB, and the line separating successes from 
failures in the secondary course by a vertical line like CD. Pupils 
in the quadrant CEB will be admit-successes; those in the 
quadrant BED reject-successes, and so on. If a pupils position 
is in cell P we know that his entrance test score was between 
79 and 80 per cent., and that his success score was only between 
40 and бо per cent. 

It is from scatter diagrams of this kind that correlations are 
calculated, and the entries usually fall within an ellipse. Along 
narrow ellipse means that a pupil with a high score on the entrance 
test generally has a high score in success, and vice versa. Тһе 
Correlation is therefore high. А broad ellipse, approaching 
Circular shape, indicates a low correlation; a long narrow ellipse 
lying along the go-go diagonal would signify a high negative 
correlation, : 

In any row or in any column the numbers are high in the 
middle and decrease towards the ends, 
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Tue Ховмлі. BIVARIATE SURFACE 


"The title is forbidding, but the notion is simple. If we were 
to erect a square tower on each cell, the height of the tower 
being proportionate to the number of the pupils in the cell, we 
should get a solid which is high towards the centre and lower 
towards the sides and ends; and if we imagine the numbers to 
be increased and the class intervals diminished, we should in the 


SCORE IN ENTRANCE TEST 


SUCCESS IN SECONDARY SCHOOL 


—Schematic representation of the bivariate surface corresponding to the 
correlation between entrance test and success. Р 


Fic. 26. 


end get a smooth surface resembling that of a policeman's helmet. 
If the distributions in the tows and columns are normal, the 
result is called a normal bivariate surface, and it has the following 
properties. Each vertical section, whether parallel to the x- or 


the y-axis, is a normal distribution curve, and each horizontal 
section is an ellipse, ? 


In view of the latter 


e + 
‘as the base of the solid. Our CEU ae and might be regarded 
is thus of the type shown in fig. 26. 


- 


GENERAL PRINCIPLES OF SELECTION TECHNIQUE 123 


When the correlation has a bivariate surface with the above- 
mentioned properties we speak of it as a normal correlation. 

AB is the pass-mark on the entrance test; CD the line dividing _ 
Successes from failures in the secondary course; O is the centre 
of the ellipse and PQ the major axis. 


Basis or THE MATHEMATICAL TREATMENT 

Our problems now become amenable to mathematical treat- 
ment, for the equation of the normal bivariate surface is known. 

If we refer to the scatter diagram in fig. 25 we see that the 
number of pupils in the quadrant CEB is the volume of the solid 
cut off by vertical planes through CE and EB. Similarly, the 
number of misfits is the sum of the volumes CEA and DEB. 
Since the equation of the surface is known these volumes can be 
determined mathematically.t 


PROBLEMS or SELECTION AND GUIDANCE 
There are three main problems at the threshold of secondary 
education: 
Prediction of the success of an individual pupil in the 
secondary course. 
Selection of pupils for a definitely limited number of scholar- 


ships or free places. A 
Selection of pupils who are fit for secondary education, the 


number not being definitely limited. 


PREDICTING THE SUCCESS OF AN INDIVIDUAL PUPIL 

То make the problem more definite we take two sample figures. 
The average correlation between an ordinary entrance examina- 
tion and success in the secondary school in four recent careful 
investigations (three in England and one in Australia) is 4:34. 
1 If OF expressed in terms of the standard deviation of x —/i, and OG expressed 


in terms of the standard deviation of y=k, then the proportion of misfits is 
given by 


1 { —eo (eo Ce ED РА Mee Г 
е ы а) | а” зас sb) 
2mV1- ri i" | nde 
References: G. Udny Yule and М. G. Kendall, An Introduction to the Theory 
of Statistics, Chapter xii. 
К. Pearson, Tables for Statisticians and Biometricians, Part ТЇ. 
First edition. London: Cambridge University Press. 
Mathematical Introduction, p. lii. 


» 
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Our correlation for the best battery, one of the highest ever pub- 
lished, is +-80. With what assurance could we, in these two cases, 
predict a pupil’s success if we were given his score at entrance? 
The methods of dealing with this problem are well known and 
are dealt with here mainly for the sake of comparison with what 
follows in regard to group selection.1 They are based upon the 
fact that if the correlation between two variates is known we can 
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— Significance of different sizes of correlation in predicting the success 
of an individual pupil. 
find the standard 


error of an estimate of the second when given 
the first.2 


if the correlation were +1, the error of our estimate 
il’s success, given his entrance test score, would be zero. 
ІҒ the error of estimate by pure guesswork 3 were 100, we сап, 
by this method, give the error of estimate for any size of correla- 


tion in the form of а percentage. "This is done graphically in 
fig. 27. 

We can now find the answers to our two questions. If the 
correlation between entrance examination and success is +34, 

1 T. L. Kelley, Statistical Metbod, Chapter viii; 
tional Measurements. New York: World Book Comp 
A. S. Otis, Statistical Metbod in Educational Measure; 
Book Company, 1925. Chapter xvii, 


а суз =01V 1 E 


3 By ‘pure guesswork’ we mean that „We predict with no knowledge of the 
pupil’s entrance test score, or on the basis of 
with success is zero. In both cases we would 


Interpretation of Educa- 
апу, 1927. Chapter vii. 
тет, New York: World 


GENERAL PRINCIPLES OF SELECTION TECHNIQUE 125 


the error of prediction of the success of an individual pupil would 
be about 94 per cent. of the error by pure guesswork. Even 
with our own highest correlation of -8 it would be 60 per cent. 

These results show the uncertainty of any guidance given to a 
pupil solely on the basis of examinations and tests, no matter how 
complete or carefully conducted. 

\ 
SELECTION FOR SCHOLARSHIPS AND FREE PLACES 

“Неге we are interested, not in the individual pupil, but in the 

number of wrong awards which we should make with different 


Systems of selection. 


4 Sample Case: Award of тоо Scholarships with 365 Candidates 


We use this as an illustration of the mathematical method; and 
we shall show how, if we know the correlation, we сап tell how 
many wrong awards we should make. 


SCHOLARSHIP EXAMINATION 


SUCCESS IN SECONDARY SCHOOL 
Fic. 28.—Award of 100 scholarships with 365 candidates. 


The pass-mark AB is fixed for us because it must cut off exactly 
тоо applicants. Since we are assuming normal correlation, we can 
easily find how far AB must be above the mean of the examination 


1 At this and various other points in the Report we have to bear in mind the 
distinction between a correlation based upon a truncated distribution and the 


correlation based upon the complete distribution. 
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distribution if it is to cut off this number.! This distance, OG, 
in terms of the standard deviation of the distribution, is denoted by 
k. -In this case OG =k = +-60. The volume ACB then indicates 
the pupils who actually get scholarships. Я 
Since these scholarships are awarded for secondary education, 
we may assume that the intention is to award them to the 100 
Pupils who would be most successful in the secondary course. 
The line dividing those who should have got scholarships бер 
those who should not is therefore а vertical line CD which cuts 4 
the top тоо as far as success is concerned. This must be "бо 
above the mean, so that OF =} = 4-66. AB and CD must meet 


on the major axis of the ellipse, and the volumes AEC and BED 
must be equal. 


We can now interpret our picture: 


The pupils in volume СЕВ got scholarships, and they come e 
the top 100 in success. "They therefore merited their awards. 

The pupils in volume AED were correctly refused clo 

The number of pupils who got scholarships in error is the 
volume AEC, 


The number of pupils who were unjustly refused scholarships 
is the volume BED. 


The number of mistakes which we should have made is the sum 
of the volumes AEC and BED. Given the size of the correlation 


it is possible to calculate all the volumes, as already explained, with 
the following results: 


TABLE XL 
Award of 100 scholarships with 365 candidates 
Method of award Correct Wrong | Wrong Total 
awards awards | refusals | mistakes 
me ааа Е р 
Pulling names out ofa hat * 2 I 

Examination : 1 5 7З = 
т= 45340. с ә 42 58 58 116 — 

r=+50 . а с 50 50 50 100 

т=+8о .. E : 69 31 31 62 


* The numbers in this row are the most probable numbers, 


1 This is done by means of Tables of the Probabilit 


ici В 252 y Integral. AES 
Tables for Statisticians aud Biometricians, Part I. - Er. Cf. K. Pearson, 


Table II. 


— PP 
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These are again disappointing results, particularly when one 
considers that few scholarship examinations, judging from the 
published results, are likely to have a correlation with success as 
high as +5. They will bring comfort only to those who at some 
time have been unsuccessful in such competitions. 


Curve showing the Significance of Correlations of Different Sizes in 
the award of Scholarships 


In dealing with the general problem it would be interesting, and 
sometimes helpful in practice, to have a curve similar to that in 
fig. 27 which would show the fall in the number of mistakes 
with increase of correlation between the scholarship examination 
and success. 

In making such a curve there is this difficulty, that the number 
of mistakes depends not merely on the size of the correlation but 
also on the ratio of the number of scholarships to the number of 
candidates. We require therefore not one curve but many—one, 
indeed, for each ratio. 

Given the ratio it is easy to make the curve. 
If T = +1, the number of mistakes is zero; if r=0, the number 
is the same as we should get by chance selection. The inter- 
mediate values can be calculated by the method described above.* 

The results for different ratios of scholarships to candidates are 
shown in fig. 29. 

The curves show that the predictive value of a correlation 
coefficient rises most rapidly when the number of scholarships 
is half the number of candidates. In this case the number of 
mistakes is reduced by one-half when the correlation is about 7. 
If the number of scholarships is very small, say *5 per cent. of the 
number of candidates, it would take a correlation of over +9 to 
effect the same reduction. 

For practical purposes a nest of different curves is not Very 
convenient, but we can make a simplification in the following way. 
By reference to the symmetry of fig. 28 we see that the number 
of mistakes would be the same when Р=Ё= - 6c as it is when 
b—k- 4.60. Further, the number of scholarships or bursaries 
awarded is not likely, at any rate at the Qualifying stage in Scot- 
land, to be less than 16 per cent., or more than 84 per cent., of the 
total number of candidates. All cases between these limits fall 

1 When the number of scholarships is half the total number of candidates 


Тһе ends are fixed. 


Я 2 4 1 5 
there is a simpler method. In this case proportion of mistakes = cos 1. 
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within the narrow band enclosed by the two lower curves. A 
curve drawn midway between them will therefore give a fairly 


NUMBER OF MISTAKES 


SIZE OF CORRELATION 


—— 50% OF GROUP AWARDED SCHOLARSHIPS 

E 16% ~ T S М 

- 23% - Е 

авола 2 Е 5 | 

Fic. 29.—Significance of different sizes of correlation п award of scholarships: 
Exact curves for different percentages. 
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Example of the Use of the Curve in fig. 30 


"То illustrate the way in which the curve could be used we take 
the sample case given above, where we know the correct answer. 
Here we have to award тоо scholarships with 365 candidates. 

"The first step is to find the number of mistakes by pure chance 
selection. This is easily done since we know that, among the first 
100 drawn from the hat, the number of incorrect awards will be 


265 
365 x100—72:6. The number of mistakes will therefore be 145. 


16 now, the scholarship examination gives a correlation of 5 
with success, we find from the graph that the number of mistakes 
will be 70 per cent. of the number by pure chance, that is, 70 per 
cent. of 145. This is ror. We shall therefore give scholarships 
to 50 pupils who should not receive them, and refuse scholarships 
to 50 pupils who deserve them. Also we know that out of the 100, 
only 5o will be correct awards. 

. These figures correspond very closely with the exact results 
given іп Table XL. If, therefore, a Director of Education knows 
approximately the correlation between a scholarship examination 
and success, he can use the graph to find the figures relating to his 
own particular problem, figures which will be more enlightening 
to his Committee than any statement about correlations. 


SELECTION OF PUPILS FIT FOR SENIOR SECONDARY EDUCATION 


It is only in regard to senior secondary education that the 
problems are likely to arise in Scotland. The treatment, however, 
is quite general and can be applied to any similar type of selection. 
1 The essential difference between this selection and the preceding 
is that the pass-mark is not fixed for us by the number of. scholarships 
or the accommodation of the schools: we have the responsibility of 
fixing it so that we shall, as far as possible, admit those who will profit 
by a senior secondary course and exclude those who would not. 


For each size of correlation there is a definite pass-mark at which 
the number of misfits is a minimum. This, as we shall show presently, 
can be demonstrated by a general proof, but for the sake of the 
non-mathematical reader we use a simpler approach. 

For the purpose of our explanation we refer to fig. 261 Let 
us suppose that 16 per cent. of the applicants are unfit for secondary 
education. The volume CAD will then be 16 per cent. of the 
total volume. If the pass-mark were fixed at AB, the number of 

2 P. 122. 


5 
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admit-fails would be the volume CEA, and the number of reject- 
successes would be the volume BED. The sum of these two 
volumes would be the number of misfits. 

If we know where AB is fixed we have all the data necessary 
for the calculation of these volumes. We can, therefore, for any 
size of correlation, make experiments to ascertain the effect of 
raising or lowering the pass-mark on the number and the com- 
position of the misfits. Table XLI is an example. 


TABLE XLI 


Effect of changing the pass-mark on the percentage of misfits 
for normal correlation 


Pass-mark at Failed Admit-fail | Reject-success | Total misfits 

Mean * 50-0 31 373 494 
= so 309 6:1 21.1 27:2 
-1'0 159 96 96 102 
E65 67 12-6 34 16-1 
-18 3:6 13-9 r6 15:6 
zig 2:9 1425 1-25 I5:50 
-20 23 14:54. 95 15-48 
-21I r8 14-79 71 15:50 
-2:6 5 155 л 15:7 
О, оо 15:9 оо 15:9 

EI 
T=+°5 Number of unfits 2 16 per cent. 


* In this chapter с denotes the standard deviation of the Qualifying scores of 
the applicants for admission to senior secondary courses. 


е жаса дат of the principles of selection are concentrated in 
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It is interesting to find that the pass-mark for minimum misfits 
is so much lower than the line separating successes from failures. 
The latter is тс below the mean, and cuts off 16 per cent. of the 
pupils; the former is 20 below the mean, and this would cut off 
less than 3 per cent. 

An important practical point is involved here. Even if we 
knew the number of fit applicants this would not enable us to fix 
the pass-mark. If we fixed it in such a way as to admit the pro- 
portion we knew to be fit, it would be much too high, and the 
number of misfits would be well above the minimum. In certain 
cases, if we wish the misfits to be a minimum we must admit far 
more than the number who are actually fit. 

We note, too, that in this particular case the number of admit- 
fails at the minimum greatly exceeds the number of reject-successes. 


At the pass-mark at which the number of misfits is a minimum, a 
pupil bas a 50 фет cent. cbance of success in the secondary courses 
below this pass-mark the pupil has more than a 50 per cent. chance 
of being a failure. This can be demonstrated in a very simple 
way from fig. 31.1 

Let CD represent the line dividing successes in the secondary 
course from failures, and let RS be the line joining the two 
horizontal tangents to the ellipse. RS passes through the centre 
of the ellipse, and on it lie the mid-points of all horizontal chords. 
Consider now the point E where RS cuts CD. If we draw through 
Е а horizontal chord AEB, AE will be equal to EB, and the volume 
of the solid on a narrow strip cut at AB will be divided into two 
equal parts at E. This means that, at AB, half the pupils are 
successes in the secondary course and half are failures. That is to 
say, a pupil with the entrance score corresponding to AB has just 
an even chance of success in the secondary course. With this 
pass-mark the number of misfits will be the sum of the volume 
CEA and BED. 

If we raise AB slightly (E being the point at which AB cuts CD), 
we shall reduce the volume CEA and increase the volume BED. 
But since, in the raised position, AE will be shorter than EB, the 
amount by which we increase the reject-successes will be. greater 
than the amount by which we reduce the admit-fails. There- 
fore, by raising the pass-mark above AB we increase the number 
of misfits. Similarly, we can show that if we lower AB below its 
present position we again increase the number of misfits. AB is 

1 Р, 132. 
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thus the point at which the misfits are a minimum, and it is the 
point at which the pupil has just an even chance of success.! 


SCORE IN ENTRANCE TEST 


SUCCESS IN SECONDARY SCHOOL 
Fic. 31.—Determination of correct pass-mark. 


We shall as before call this point at which the number of misfits 
is a minimum the correct pass-mark. 


Curves of Misfits 


If we know the correlation between an entrance test and success, 
we can, for any proportion of unfits in the group of candidates, 
calculate the number of misfits for any pass-mark. ‘Thus we can 


plot a graph which will show in a clearer way the effect of raising 
or lowering the pass-mark. 


We can also do this for different sizes of correlation, and see how 
the curves compare with one another. 


Such curves are given in fig. 32, a diagram which should be 
studied by all who have to deal with selection. It is based on a 


proportion of fits to unfits which is very close to that found in the 


2 The minimum is always at the point where the regression line of the 
X-arrays cuts the line dividing successes from failures. For the mathe- 

- matician, who has read the above popular demonstration with impatience, it 
may be added that a proper mathematical proof is easy. We take ies PEOR 
for the sum of the volume CEA and BED as given in footnote Р. 123, E 'en- 
tiate and equate to zero. Mr A. S. Robertson, pitis 


who carried out thi 4 
found it to be much less difficult than might be expected, € operation, 
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City in which the Inquiry was conducted, and is likely to be 
reasonably similar to that in other parts of Scotland. We take it 
that about 72:6 per cent. of the applicants are fit for secondary 
education and 274 per cent. are unfit. Curves for four sizes of 
correlation are shown. 


PERCENTAGE OF MISFITS 


CURVE OF 
— — — CORRECT 
PASS MARKS 


PASS MARK (о) 
Fro. 32.— Curves of misfits for different sizes of correlation. 
(Proportion of unfit less than half.) 

If tbe pass-mark is raised above tbe correct point tbe number of misfits 
rises to a maximum equal to the number of fit applicants: if lowered 
below the correct point, to a maximum equal to the number who are unfit. 

"This is obvious with or without the help of fig. 32. 
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If the number of fit applicants is greater than the number of unfit, 
the number of misfits rises more quickly when the pass-mark is raised 
above the correct point than when it is lowered below it. This principle 
is clear from a study of the curves in fig. 32, and it will be seen 
that the effect is more pronounced with the lower correlations. 


Tt is an important one, for it shows that the danger, in Scottish : 


PERCENTAGE OF MISFITS 


"on mg wu 
Thà > 
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CURVE OF 
CORRECT 
PASS MARKS 


PASS MARK (о) 
Fic. 33.— Curves of misfits for different sizes of correlation. 
(Proportion of unfit one-half.) 
conditions, lies on the s 
side of a low one. 
We here come u 
Nearly 


in fig. 31 cuts the distribution, that is, 


ide of a high pass-mark rather than on the 
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Fig. h 
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cent., the curves would be the mirror images of those in fig. 32, © 
and the misfits would rise more steeply on the side of lowering 
the pass-mark. 

This brings out the important fact that, in considering the 
problems which arise in selective examinations, such as the fixing 
of pass-marks, the Education Committee must adapt its procedure 
to the percentage of unfits in the group of candidates. There is, 
of course, no force in the objection that this could not be known 
unless all the candidates went on to the secondary course; and 
that, even then, the information would be available three years 
too late, All that is needed is a general idea of the proportion, 
and every Director of Education is in a position to know this. 
In Scotland, for example, the proportion of unfit candidates for 
secondary education will almost certainly always be less than 50 
percent. In the City in which the Inquiry was made it is about 
29 per cent., and varies little from year to year. Fig. 32 is there- 
fore the case that is relevant to Scottish conditions. 


The practical value of an entrance examination depends not 
merely upon its correlation with later success but also upon the 
percentage of unfit candidates. By practical value we mean the 
value to the Education Committee in ensuring that the number 
of misfits is a minimum. 

The truth of the principle can be seen by a comparison of 
figs. 32 and 33, from which we see that an examination giving 
a correlation of +5 with success might be of some use if half the 
candidates were unfit, but that it would be almost useless if only 
27 per cent. were unfit. 


If the percentage of unfit candidates is under 30 it is probable 
that many of the present entrance examinations are a waste 
of time and money. The principle is stated in a provocative 
way, but the following are the facts upon which it is based. We 
have already pointed out that the average correlation between 
entrance examination and success recorded in four published 
investigations was :34; and, judging from the data available, 
we doubt whether most entrance examinations have a correlation 
as high as -4. 

If that is not so, our principle has to be modified. But, taking 
first an examination with a correlation of :5, we see from fig. 32 
that the minimum misfits, if we used the correct pass-mark, 
would be 25 per cent. If we used no examination at all, and 
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admitted all the candidates, the number would be 27:43 per cent.: 
the saving of misfits would thus not be very great, even if we 
managed to hit the exact pass-mark. But if, as in our experience 
very often happens, the pass-mark were too high, say —:5o 
instead of — 1-20, the number of misfits would be 28-34 per cent.; 
that is, more than we should have if we admitted all the candidates. 

If the correlation were +-4, the minimum misfits with the 
examination would be 26:55 per cent.; with a correlation of 
3:3 it would be 27-23 per cent. ; 

This does not mean that an Education Committee should 
give up its entrance examination—merely that it should take 
steps to find out the correlation 


which the examination gives 
with success and the normal 


percentage of unfits among its 
candidates. If the results, considered in the light of what has 
been said above, show that the examination is nearly valueless, 
it should face the facts and mend its ways. This can be done 
by improving the setting, administration and correction of the 
examination, and by adopting some of our later recommendations 
as to the inclusion of other measures in the battery. 

We leave as an exercise to the reader the consideration of the 
case where the percentage of Jit candidates is 30 per cent. or less 
and the correlation is about 74, hoping that he will not conclude 
that the best plan would be to reject the lot—thus saving not 


only the cost of the examination but also that of the secondary 
education, 


If the number of fit candidates is greater than tbe number of unfit, 


then the higher the correlation between the entrance examination 
and success the higher is the correct pass-mark. "This important 
result is clear from fig. 32. 'The point at which we get the 
minimum of misfits is lower for the lower correlations, the law 
of fall being indicated by the broken curve. The difference is 
Not very great for correlations between 775 and 1-0, but below -5 
the correct pass-mark falls very rapidly, à 

Аз usual we find the neutral point when 50 per cent. are unfit. 
The correct pass-mark is then always the Same, as can be seen 
from fig. 33. unfit pupils is more than 50 


4 
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the law is reversed, and the higher the correlation the lower 
should be the pass-mark. 

In fig. 34 we give the exact curve of fall for the case where 
27:4 per cent. of the candidates are unfit, and from this a Director 
of Education can get a rough idea as to where his pass-mark 
should be drawn if he can estimate the size of the correlation 
between entrance test and success. 


CORRECT PASS MARK (о) 
one 
| 
E 
|. | 
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SIZE OF CORRELATION 
Fie. 34.— Correct pass-mark for different sizes of correlation. 
(Percentage of unfit 27-4.) 

If the number of fit candidates is greater than the number of 
unfit, the pass-mark should never be fixed above the point at which 
it will cut off a number equal to the number of unfit. It can easily 
be demonstrated that if the line separating the fits from the 
unfits is at — ро, then the pass-mark will be highest when 7 — 170, 
and it will then be at — 2c. 

'The rule may serve as a rough guide to a Director of Education. 
If he estimates, from the experience of years, that the percentage 
of unfit candidates is about 27, he knows that the pass-mark 

5* 


should certainly not be above this point, unless the distribution 
differs considerably from the normal form. 
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will be greater than the number of rejeci-successes. The purport 
of this principle is that, if an Education Committee adopts our 
suggestion as to the correct pass-mark, it is not erring on the 
side of refusing to give the child his chance. 


The further the percentage of unfit candidates falls below 50 
the greater will be the preponderance of admit-fails over reject- 
successes at the correct pass-mark. The size of this effect, for a 
correlation of + :50, is shown below: 


Кегсепеде Percentage at the correct pass-mark 

candidates Admit-fails Reject-successes Misfits 
50-0 16-7 167 353 
308 211 бі 272 
159 145 ae 155 
115 196) 3 114 


If the number of fit candidates is greater than the number of unfit, 
the lower the correlation between the entrance test and success, the 
greater is the preponderance of admit-fails over reject-successes at 
the correct pass-mark. In fig. 35 we have plotted the curves 
of admit-fails and reject-successes at the correct pass-marks for 
different sizes of correlation. ‘The curves are in respect of the 
case where 274 per cent. of the candidates are unfit, which is 
nearly the normal proportion in the City in which the Inquiry 
was made. 

This shows that, under Scottish conditions, the poorer the 
entrance test the more we err on the side of giving the pupil his 
chance, provided we use the correct pass-mark. 


If the number of fit candidates is greater than the number of unfit, 
the lower ‘the correlation between the entrance test and success the 
greater the number of pupils we have to admit. This is an important 
practical law, for it has financial implications. We assume, of 
course, that the Education Committee desires to have the misfits 
a minimum; and, if so, it is clear that it is bad economy to save 
money on the entrance testing. 

Taking what we have regarded as a fairly normal percentage 
of unfit candidates for Scotland, that is 27-4 per cent., the per- 
centage of candidates that we should have to admit for different 
sizes of correlation between entrance test and success is as follows: 
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Fic. 35.— Curves showing relative pro: 


‹ portions of admit-fails and reject-successes 
at the correct pass-mark for different siz 


es of correlation. (Percentage of unfit 27-4). 

From this it is clear that, if we are to deal fairly with the 
pupils and make the most economical use of the national resources 
of money and talent, we must admit to the senior secondary 
schools more pupils than the number who are fit. If we had a 
perfect test and used the correct pass-mark, we should admit 726 
per cent.; with our own best battery (© =-8), we should have to 
admit 77:3 per cent. With an examination, giving a correlation 
of about “4, we should have to admit 93:3 percent. If there were 
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1,000 candidates this would mean that we should have to provide 
secondary education for 160 more pupils, a large proportion of 
whom would be failures. And it will be clear by now that an 
Education Committee cannot counteract the effects of a poor 
examination by using a higher pass-mark than the correct one for 
that examination. 


If about 27 per cent. of the candidates are unfit, and if the number 
of failures in the secondary course is less than Q per cent. of the 
number admitted, the system of selection is unsound. This is merely 
a rough rule for Scottish conditions, and it draws attention to 
an erroneous opinion that is sometimes held. Some Education 
Committees, in experimenting with different systems of selection, 
collect statistics as to the success or failure of the pupils admitted 
to the senior secondary schools; and in discriminating between 
such systems it is occasionally assumed that the one producing the 
fewest failures in the secondary school is the best. ‘This is far 
from the truth. The fact that system A admits fewer failures 
than system B probably means that A is the less sound system; but 
the position is that we cannot judge on this basis alone. We must 
take into account the correlation, the proportion of unfit candidates 
and the position of the pass-mark. 

We have already established that it is vain to aspire to a system 
with no misfits. "The question then arises as to what should be 
the percentage of failures among the pupils admitted if the system 
is as sound as present conditions permit. Our answer depends on 
the percentage of unfit candidates; but, taking this to be approxi- 
mately 27 per cent., and assuming normal correlation, there should 
be 14 per cent. of failures among those admitted. If the dis- 
tribution had a tail of weak pupils, as in our Inquiry group, 11:5 per 
cent.! of those admitted should turn out failures. These figures 
are based on our best battery; and since the minimum number of 
admit-fails would be greater with a poorer one, it will be seen that 
our figure of g per cent. is conservative. 


Curve showing Predictive Significance of Correlations in the Selection 
of pupils for Secondary Education 
The nature of the curve depends on the proportion of unfit 
candidates; but if we take this to be 27:4 per cent. of the 
total, we obtain a result applicable with reasonable accuracy 


1 Tf we used the additional principles mentioned in Chapter XIV the figure 
would be то-т per cent. See also Chapter XVII, p. 173. 
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to Scottish conditions. Fig. 36 gives the minimum number of | 
misfits for each size of correlation as a percentage of the minimum à 4 
1 


! 


number for zero correlation; that is, it assumes that the correct 
pass-mark is always used. 
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Fic. 36.—Significance of different sizes of correlation in selection of pupils fit 
for secondary education. 
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CHAPTER XV 


ATTEMPT TO IMPROVE SELECTION BY SPECIAL 
CONSIDERATION OF BORDER-LINE CASES 


So far we have assumed a hard-and-fast pass-line. We now 
consider whether a careful scrutiny of border-line cases might 
justify the admission of certain pupils just below the pass-mark or 
the rejection of some just above it, thus improving our selection. 
One advantage of the misfits technique is that we can make 
experiments on such problems with an ease and speed much 
appreciated after the laborious correlation procedure. 


кітілі, STANDARD Момвев or Misrrrs 
For the purpose of the experiments which follow we use the 
complete battery IQ +Q + S + Ts, for which the minimum misfits 
in the senior secondary group are as follows: 
Admit-fails Reject-successes Misfits 
39 20 59 


Number in group 
452 
"These, then, are our standard figures in relation to which we 
assess the value of any próposed principle. 


SUPPLEMENTARY PRINCIPLES INVOLVING ОЕ ОЕ TEST 
AND EXAMINATION SCORES 


IMPOSING A MINIMUM IN EACH SUBJECT FOR ALL PUPILS 


The subjects are intelligence, English and Arithmetic. English 
is taken to be Qe +E + Tes, and Arithmetic Qa +A + Tas. By 
imposing a minimum of o sigma we mean that a pupil is rejected 
if he falls below o sigma in any subject, no matter how high his 
average score may be. 

143 
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We give below the numbers of misfits with different minima: 


Minimum Admit-fals — Reject-successes Misfits 
+ -6 sigma 20 62 82 
ғ“ 26 33 59 
+2 32 26 58 
оо 34 22 56 
= +2 38 21 59 
— ч 39 20 59 
аа 75 39 20 59 
Standard figures 39 20 59 


In interpreting this and the following figures we have to bear 
in mind the difficulty that, while the complete group is reason- 
ably large, the numbers affected by a principle may be small. 

here is no satisfactory escape from this; but we can introduce 
some of the steadines 
below by using graphs. By examination of the sweep of Ta 
after smoothing, we can usually tell whether or not a critica 
figure is likely to be accidental. In the above case when we 
plot misfits against the minima we find confirmation that there 
is likely to be a minimum at o sigma, but not quite so low 


as 56. 
With this 


general reservation we hazard the following con- 
clusions: 


A small reduction in the 
imposing a minimum in the separate subjects. 


The minimum has to be very exactly determined: if it is too 
low it gives no improvement; if it is too high the number 
of misfits is greatly increased 


number of misfits can be made by 


The practical import is that the 
if the minimum is Scientifically de 
we shall explain later. 


principle should be used only 
termined by a method which 


IMPOSING A MINIMUM IN EACH SUBJECT FOR WEAKER 
PUPILS ONLY 


To impose the minimum only on pupils with average scores of 
+1-0 sigma and under is the same as to demand a. higher 
pass-mark, namely, +1-0 sigma, if the pupil falls below the 


з which comes from the numbers above and - 


» 
1 


> 


BORDER-LINE CASES 145 


minimum in any part of the examination. The numbers of 
misfits with different minima are shown below: 


Admit- Reject- Misfits 


Mini i 7 

inimum On pupils below fails EE 
0-0 sigma +1-5 sigma 34 22 56 
41-0 35 22 57 
ми 37 22 59 
+ 8 37 21 58 
On all pupils 34 22 5 
+ +2 sigma TIO 34 24 58 
+ 8 36 22 58 
On all pupils 32 26 58 
+ 53 sigma ` IO 31 26 - 57 
= COME и 
On all pupils 31 28 59 


Clearly nothing is gained by limiting the application of the 
minimum to the weaker pupils. 


IMPOSING A MINIMUM (а) ім 10 ONLY, (0) IN ENGLISH AND 
ARITHMETIC SEPARATELY BUT NOT IN IQ 


We found that both of these gave poorer results than the 
imposition of a minimum in each subject as in the preceding 


sections. 


ALLOWING A LOWER PASS-MARK IF THE PUPIL HAS 
HIGH ABILITY IN ONE SUBJECT 


This is the converse type of allowance. It is based on the 
idea that a pupil who falls just below the pass-mark but has a 
very high IQ or very high ability in English or Arithmetic, may 
have a better chance of success in the secondary course than a 
pupil who is just above the border-line in all subjects. The 
number of misfits when a lower pass-mark is allowed for pupils 
with high ability in any subject is as follows: 


А 
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E Low z 5 

ability allowed ails successes 
Over +1-3 sigma +-6 sigma 39 19 58 
+5 40 19 59 
+4 19 41 6o 
+3 19 41 бо 
Over +1-1 sigma T6 I9 42 61 
+5 18 43 61 
+4 18 45 3 
#53 18 45 63 
Standard figures 39 20 59 


It is therefore most unlikely that any great improvement could 
be made in this way. Our results show a slight reduction if we 
allow а pass-mark -1 sigma lower for pupils who are above 
+173 sigma in any one subject, 


By this procedure we find that the following are the numbers 
of misfits: 


Admit-fails Reject-successes Misfits 
34 21 55 
The resulting improvement (4 misfits out of 


and the above now become our standard figures 
of further principles, 


59) is considerable; 
for the application 
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PROTECTING THE PUPIL AGAINST A FLOP IN ONE TEST 
OR EXAMINATION 


We have given an example of this in fig. I9, where it is 
clear that some accidental disaster has overtaken the pupil in 
the Arithmetic examination. In diagnosing a flop it is desirable 
that we should have at least two other reliable measures in the 
same subject; and the drop should be greater than those that 
could be explained by normal fluctuations of performance and 
marking. ' 

We cannot apply this principle to a low IQ as we have no 
other tests in the same subject to act as a standard. Yet with 
many pupils we had the strongest suspicion that the IQ result 
was of the flop type, and that this increased the number of misfits, 
We therefore make two suggestions. It is desirable that further 
research be undertaken towards obtaining comparable IOs from 
different group tests. It would then be an advantage if Education 
Committees were to arrange for the application of a group or 
individual intelligence test to all primary pupils at least a year 
before the Qualifying stage. The results of such testing could 
be used to protect the child against the danger to which we have 
referred, and could be put to many other uses. Another method 
would be to apply an individual test in all suspected cases. 

After some experiment it was found that the best system was 
to count a score a flop if: 

(1) The other two scores in the same subject were within 
*3 sigma of each other. 

(2) The third score was at least “7 sigma below the lower of 
the other two. 


Тһе effect of protecting against a flop in one test is shown 
below: 
Admit-fails Reject-successes Misfits 
After allowance for flops 34 19 55 
Standard figures . s 34 21 55 


'The principle thus makes a reduction of 2 misfits in 55; and, 
while the numbers are so small that we cannot vouch for the 
result with any great degree of statistical assurance, our experience 
of several years of practical selection warrants the recommendation 
that it should be accepted. 

It will be understood that the number of flops in our group 
was much greater than 2. The difficulty, indeed, with all these 
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Subsidiary principles is that they are two-edged weapons. A 
number of pupils who were satisfactorily housed with the reject- 
fail group, and whom we should have been only too glad to leave 


there, were transferred by our flop principle to the admit-fails. 
They thus increased the number of misfits. 


SUPPLEMENTARY PRINCIPLES INVOLVING Use or 
Отнек Data 


We now consider whether we could improve our selection by 
going beyond the test and examination results and taking into 


account such factors as health, industry, home conditions and 
personal qualities. ` 


ALLOWANCE FOR HEALTH 


It is unlikely that anyone, outside ultra-scientific circles, would 


think of including health ratings in the scores before the pass-mark 
is determined. The most that we could do would be to také 
it into account in extreme gradings or border-line cases, and one 
method would be to reject all who have a very low health grading, 
irrespective of what their average score may be. То test this we 
tried rejecting all with a health grading of E, and found that we 


increased the misfits by 1; when we rejected all with D or E 
the increase was 1 5. 


Another method 
border-line cases only. 
but had a health grading 


: lotting age-allowances, We gave a bonus 
of -2 sigma for an A, 


Health allowance Admit-fails 


Reject-successes Misfits 
Ac : B+-2, etc. 43 I 6 
A42 :В+чл,ес. . 37 19 56 
Atr :В- оф, ес. . 37 19 56 
А--о2: B+-o1, еіс. . 35 19 54. 
Standard figures. 24 I9 
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The conclusion is only too obvious, and is confirmed by the 


method mentioned on p. 144. If we plot number of misfits 
against size of health allowance the curve reaches the minimum 


when the health allowance is zero. 


ALLOWANCE FOR INDUSTRY 


i Fig. 37 will illustrate the temptations to which one is exposed 
in dealing with these Qualifying problems. The correct pass-mark 


e ar 
IQ ае E Tes ба A Tas Av Tp Ind. H Fc. 


Fic. 37.— Case where allowance for industry, etc., would lead to error. 


2 
TI ILES 


is +70, and this girl's average score is 4:69; yet she has very 
pronounced practical ability; is well above the average in industry 
and in health; has originality and special talents in Art and Music; 
comes from a good home; and her primary head master is 
confident that she will be a success. Surely we were intended 
to admit this girl and give her the chance of a senior secondary 
education which may be the road to a career in music or art. 
We should in this case have been wrong. 

In fig. 38 we give the opposite type of case, where we should 
have been saved from error by taking into account the teacher's 


forecast and industry grading. 
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As with health, we first rejected all pupils with an E in industry. 
This made no change in the number or composition of the misfits. 
Rejection of all with D or E ratings left the total number of 
misfits unchanged, but slightly lowered the number of admit- 
fails relative to the reject-successes. We also tried the method 
of assigning a bonus or penalty in accordance with the grading, 


ІЗ; Qe E Tes Qa A Tas Av. Tp МаН Ес T аг Hess 


Fic. 38.— Case where allowance for industry, 


ctc., would improve selection. 


thus taking industry into account on 
effect of such allowances for indust: 
is shown below: 


ly in border-line cases. The 
ry on the number of misfits 


Industry allowance Admit-fails Reject-successes Misfits 
А+то : В +:5, ес. . 49 I2 61 
A+ 8 :В+д,ес. . 45 12 57 
A+ 6 :B+-3, etc. . 43 14 57 
А+: B+-2, ес. . 4I 14 55 
A+ 2:В-ьч,ес. . 40 16 56 
A+ зт :В+:05, ес.. 39 18 57 
A+ -02: B+-o1, etc. . 36 19 55 

Standard figures . 34 
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. We can only conclude that nothing is to be gained, even 
in border-line cases, by taking account of industry gradings as 
obtained in our Inquiry. Perhaps one reason for this is that 
the pupil’s industry will be reflected in his test and examination 
results and is thus already taken into account. 


ALLOWANCE FOR TEACHERS' FORECASTS 


On the rating cards the primary teachers gave forecasts of the 
pupils’ success in each of the four main types of post-primary 


Ind.H Fe I E Ш 5 


IQ Qe E Tes да А Tas Ау. Tp 


Fie. 59.— Case where allowance for teacher's forecast would lead to error. 


course. А five-point scale was used, in which A indicated 
distinguished success, B clear success, C doubtful success, and 
so on. 

We had high hopes of reducing the misfits through their use, 
for we noted frequently that pupils whose success was greater 
than we should have expected from their test results had forecasts 
of A or В. But once again we had a compensating number of 
cases, like that shown in fig. 39, where the forecast was clearly 


wrong. 
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This pupil has an average score of +-72 sigma and therefore falls 
just above the pass-mark. But her primary teacher reports that 
she is untidy, slow in speech, movement and learning, and grades 
her as a hopeless failure in a senior secondary course. Surely, 
on such additional information we should bring her below the 
line. 'The right-hand side of the profile shows that, had we 
done so, we should have made a decided blunder. The pupil 
took a one-language course, attained nearly the average of the 
secondary group in her first two years, and showed signs of 
improvement even on that satisfactory standing. А 

By failing all pupils with a teacher's forecast of E we increased 
the misfits by 2, while rejection of all with forecasts of D or E 
gave an increase of 4. | 


Using the forecasts in border-line cases only we get the follow- 
ing results: 


Forecast allowance Admit-fails Reject-successes Misfits 
Acro :B4s ес. . 41 18 59 
A+ 8:В-4, ес. . 40 22 62 
A+ 6 : B+-3, etc. . 40 21 бі 
А+ 4 :В+-2, etc. . 39 20 59 
A+ -2 : B+, ес. . 39 20 59 
А+ то: B+-05, etc. . 37 21 58 
A+ :02: B+-0o1, etc. . 36 20 56 

Standard figures . 34 19 53 


The conclusion is that the use of the teachers’ forecasts even 
in border-line cases increases the number of misfits. 


ALLOWANCE FOR PERSONAL QUALITIES 


Primary teachers were asked to note any marked personal 
qualities of the pupils. This was not a very scientific way of 
collecting the data, but we felt that it would be sufficient for a 
preliminary exploration of the problem. 

The view has often been expressed that existing systems of 
selection are based too exclusively upon intellect and scholastic 
attainment, and that success їп higher education may be dependent 
not merely upon these but also upon certain personal qualities. 
At first sight our case histories seemed to bear this out. In 
reviewing the profiles we encountered a number where a pupil 
just under the admission line had a number of good qualities and 
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proved a success; and conversely where a pupil just above it 
had marked bad qualities and was a failure. The matter was 
therefore worthy of further study. 

To reduce the results to a form amenable to statistical treatment 
was far from easy, but the following rough procedure was finally 
adopted. We divided the qualities into three categories, good, 
bad and neutral. In so doing we were guided largely by common 
sense, and partly by the results of previous studies, which suggest 
that the most important qualities from this point of view are 
those like logical and critical ability, independence of thought, 
eagerness to learn, and are not to be found on the side of emotional 
life and character. In the good category we included such 
qualities as perseverance, reliability, earnestness, ambition; in the 
bad group, carelessness, indifference, laziness, ready discourage- 
ment, and so on. To the neutral category we relegated all 
those that seemed to be irrelevant to success in the secondary 
course, for example, amiability, distinct charm of manner, good 
nature, sensitiveness, timidity. In order to take the qualities 
into account in border-line cases we gave a bonus for each good 
quality and a penalty for each bad one. The effect of these allow- 


ances on the number of misfits is shown below: 


Bonus or penalty f i 
y for "n ject- 
personal qualities ошо Soe БА. 47 мч 
3 sigma 39 18 57 
D 38 20 58 
i 39 19 58 
05 37 18 2355 
Standard figures 34 19 53 


Once again we get the disappointing result that, however well 

the idea may work in individual cases, it is not a success when 
generally applied. 1% therefore, personal qualities are to be used 
in selection, the data must be obtained by a better method than 
that used by us. 
_ There is, however, a reason why the use of personal qualities 
in selection is unlikely to make any great improvement; as with 
industry, the qualities which matter are already reflected in the 
pupil’s test and examination results. 


1 О. Bobertag, Schéilerauslese. Berlin, 1934. Р. 64. Privately printed for 
the International Examination Inquiry. á 
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ALLOWANCE FOR HOME CONDITIONS 


By this time we had abandoned all hope of improvement 
through the use of such supplementary data, but for the sake of 
completeness we felt we should not leave this last stone unturned. 
We divided the reports on home conditions into good, bad and 


neutral, and examined the effect of taking these into account at 
the border-line: 


Bonus or penalty for 


52% Admit-fails Reject-successes Misfits 

home conditions 

"4 sigma 44 I5 59 

3 ч 40 17 57 

2 41 18 * 59 

I 37 18 55 

95 37 19 56 
Standard figures 34 19 53 


Despite this negative result, which may be due to our method 
of collecting the data, we feel that home conditions do have an 


important bearing upon a child's success in the secondary school, 
and we shall return to the point at a later stage. 


ALLOWANCE FOR COMBINATIONS OF THE SUPPLEMENTARY DATA 


It is just possible that, while we secure no improvement by 
taking the additional factors separately, we might do so by taking 
them together. One pupil might, for instance, survive with an E 
in industry, another with an E in health, another with extremely 
bad home conditions. But if the same border-line pupil had all 


three we might be justified in prophesying that he would be a 
failure. 


We tried yarious combinations without a 
ment; and finally we included every ratin. 
the allowance made was that giving the least unsatisfactory result 


when it was taken separately, and an age allowance was added. 
The result was that the total number of misfits was increased from 
the standard figure of 53 to 61. 


ny resulting improve- 
g. For each measure 


~~ 
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MORE SCIENTIFIC SYSTEMS OF MAKING ALLOWANCES 


So far we have used the straightforward system of giving a bonus 
for an A or a B and a penalty for a D or an E. This is probably 
the only system ever likely to be used in practice, but it has a 
defect which may be indicated by the following figure: 


1 1 1 1 1 
Е р C ЖЕКЕ А 
o +1-07 +2:20 


—2:20 sigma -1:07 


Fic. 40.—The need for a more scientific system of making allowances. 


„Оп the assumption that the industry gradings are normally 
distributed, the means of the categories ‘A, B, C, D and E would 
be as shown. If, at C, the pupil had a 50 per cent. chance of 
success, the system which we have used would be sound; but if 
the point for an even chance of success were at Е (that is, if a pupil 
had to be more industrious than the average to have an even 
chance of success in the secondary school), a C grading should get a 
penalty, and the bonus for a B would be less than the penalty for a 
D. A more scientific system would therefore make the bonuses 
and penalties proportional to the distances of the means of the 


categories from Е. 


Industry 

Тһе data determining the point at which the pup 
chance of success have some interest in themselves, 
below: 


il has an even 
and are given 


Industry grading Successes Failures Total toe = f 
A 52 2 54 3:70 
B 192 29 221 1312 
(е; 73 76 149 SIOL 
D 2 20 22 90:91 
E 2 5 5 100:00 


"There is a clear correlation between industry and success; and 


the point for an even chance comes out to be --15 sigma. Аз 
the mean of C category is —-18 sigma, the straightforward system 


of bonuses is here justifiable. 
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Teachers Forecasts 


The percentage of failures in the various categories of the 
teachers’ forecasts are as follows: 


age 
Forecast grading Successes Failures Total Ec 
d 92 2 54 3:70 
B тщ, 30 244 12:30 
© qr 59 100 59:00 
> В $1 39 7949 
р E 19 12 83:33 


There is again evidence of correlation, but it is interesting to 
note that nearly 4 per cent. of those whom the teachers expected 
to be distinguished successes have actually proved to be failures, 
and that about 17 per cent. of those for whom they predicted 
hopeless failure have been successful. By the time the reader has 
finished this Report he will, we trust, fully realise that this is no 
reflection upon the care or judgment of the teachers. 

At this point we interpolate a Table prepared by Mr Robert 


Robertson, M.A., which throws additional light upon the reliability 
of teachers’ forecasts, 


TABLE XLII 


Teachers’ forecasts of success in senior secondary course for 
different groups 


el 


Percentage 


Group N 


Senior secondary 452 | 12-1 | 54-6 | 21:8 | 88 27 
Junior secondary Е 


р - | 1,930 6 | 17- б E 14 
Special classes for backward ГАТ S ina 
children 2 . 509 | o 2 


79 | 200 | 78:9 


'These data show that the prima 


. ty teachers can make а fair 
sifting of the material at the О 


ualifying stage, but that we could 
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not fely solely upon their estimates. If, in the senior secondary 
group, we had used the teachers’ estimates alone, we should have 
had 42 misfits in the A, B, D and E categories; and, in addition, 
тоо pupils were graded С. 

Finding that the point for an even chance is 4:20 sigma, we 
constructed two scientific bonus systems, in one of which A got 
a bonus of +:20 sigma, and in the other +-10 sigma. Neither 
was of any avail, the total number of misfits remaining obstinately 
the same, though the composition was considerably altered. 


Health and Personal Qualities 

For these factors the correlation is so small that the point for an 
even chance of success cannot be determined. ‘The percentage of 
failures in thevarious groups of personal qualities is given merely for 
its general interest to any who may contemplate a more scientific 
attack on this problem: 


Personal qualities Successes Failures Total aye ERIS 
3 good qualities ` 24 4 28 143 
2 good qualities 61 5 66 76 
I good quality . 4 79 16 95 168 
No marked qualities . 156 79 215 367 
1 bad quality 18 26 44 591 
2 4 50-0 


2 bad qualities . 5 2 


AGE ALLOWANCES 


We give some results bearing upon one particular aspect of this 


much-discussed topic. 


Relevance of tbe Age-Allowance Procedure to Scottish Conditions 


The introduction of age allowance usually springs from a desire 


to be fair to all candidates in the competition for a limited number 
of scholarships or places in secondary schools. If the examination 
is confined to pupils between 11 and 12 years, it is felt that the pupil 
of 12 has an unfair advantage over the pupil of 11. The younger 
child should therefore be assigned a bonus; and the test of fairness 


is that the percentage of awards should be uniform throughout the 


age-range. 


Morally the idea is excellent; and it is a pity that the need for it 
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should arise іп this system of limited competition whose educational 
morality is open to question. Even here a wider view of fairness 
is probably necessary; and it is certainly so in Scottish conditions 
at the Qualifying stage. Our bursaries are intended to meet the 


competition, it should be limited only by this ability to profit, not 
by a fixed number of bursaries or places in secondary schools; 
and it follows that our test of fairness must take the child’s success 
in the secondary course into account, If we admit a young pupil, 


by giving him a bonus for his youth, our fairness may be costly 
to him should he Prove to be a failure, 


to about I per cent. The following are the effects of different age 
allowances on the number and composition of the misfits: 


Age allowance per month Admit-fails Reject-successes Misfits 


"IO sigma 49 37 86 
505 42 27 69 
"04 39 24 63 
°з 39 24 63 
“02 36 21 57 
191 37 20 57 
2 39 20 59 
-703 44. 22 66 
495 56 31 87 
ДО 74 45 119 


Standard figures 


will be seen that the general tendency of 


the curve is clear. The 
negative allowances are, of Course, 


merely of the nature of a mathe- 
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matical diversion. They mean that we give the old pupil a bonus 
simply for being old, and that we penalise the young pupil for his 
youth. Even with as modest an allowance as З per cent. a month 
we increase the number of misfits, and the higher the allowance the 
greater is the increase. Ап allowance of 1 per cent. a month would 
be almost as harmful as a negative allowance of -5 per cent. 


NUMBER OF MISFITS 


-:04 —02 О +02 +04 +:06 +08+1 


AGE ALLOWANCE PER MONTH 
(sigma) 
Fic. 41.—Effect of different age allowances on the number of misfits, 


Effect of more Scientific Age Allowances on the Number of Misfits 
Certain defects of the straightforward type of age allowance are 
brought out by figures І, 2 and 3,1 which show that the age 
allowance for each month should not be the same for the weak 
as for the strong pupils. A scientific allowance should therefore 
depend on the pupil’s score.? 
From our data we prepared the following differential allowance: 


Pupil’s score Age allowance per month 
Over +1-32 sigma 4-:03 sigma 
+:50 to 41-31 +04 
—:50 to + 49 4-05 
Below --5о 4-03 
1 Pp. ro-11, 


* G. Н. Thomson, “Тһе Standardisation of Group Tests,” British Journal of 
Psychology, vol. ii, Parts II and III. J. B. T. Davies and G. A. Jones, The 
Selection of Children Jor Secondary Education. London: George С. Harrap & 
Co., Ltd., 1936. Chapter vii. 
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5 Е = mo hn 
With this system the number of misfits is increased fro 
53 to 66. | р . | . Ис. 

"These results should Ье considered in conjunction with or 
given earlier in regard to correlations with age, the high ее а Ж 
between intelligence score and intelligence quotient, и the C 
agreement between the follow-up correlations for these. 


Summary 


This has been a depressing chapter of negative results and E 
carded principles; yet it has cleared the air, and others may pro 1 
by our mistakes. For the benefit of future investigators we m 
emphasise one important point. A study of these supplementa 7 
factors in border-line cases can be most satisfactorily made if " 
selection system has been operative. In this we were аҚ Sin "i 
fortunate, for our Inquiry deals with the last group admitte 
senior-secondary schools without a fairly strict selection. zd 

Despite our disappointments, we have gathered a crop of 6; 2 of 
a saving of 6 misfits out of 59 is worth making. The cor 
the complete battery with success was "800; if we use the supp. 
mentary principles the result is equivalent to a correlation of "854 

The only additional principles which we can recommend are: 

The imposition of a minimum for each separate subject. 

The allowance of а slightly lower pass-mark if the pupil shows 
high ability in one subject. 

The protection of the pupil against the effects of a flop in one 
subject. 3 


We shall show later how these principles could be used in practice. 


T success in reducing the number of misfits m 
© group numbered 452, of whom 320 were 
for secondary education: 


Method of selection 


Number of misfits 
АП applicants admitted 


z 4 132 
Complete battery (r= +8) with pass-mark 

ОЁ +-7 sigma - D 3 3 59 
The same, with above additional principles 


53 
1 See Note by Professor Godfrey H. Thomson on р. 161. 


AGE ALLOWANCES 
Nore sy Prorrssor Goprrey Н. Tuomson 


This fact, that giving age allowances increases the number of 
misfits in the senior secondary courses, is in agreement with what 
I too have sometimes found in researches in England, namely 
that entrance examination marks without age allowances correlated 
slightly better with secondary school performance than did marks 
with age allowances. These results might appear to be an argu- 
Ment against age allowances. But I do not draw that conclusion, 
for the object of an age allowance is not to improve prediction but to do 
justice to children born in different months of the year. 

It is true that age allowances, at any rate at first sight, seem to be 
less imperatively necessary at this stage in Scotland than in England. 

‘In England the examination is more competitive, and is almost 
invariably held only once a year, whereas in some Areas in Scotland 
it is held every six months. In England a pupil as a rule must sit 
the examination with his age-group, and save in exceptional 
circumstances has no further chance; in Scotland he may wait 
until a later age, and may even then be allowed to sit again if 
unsuccessful. 

But these differences are more apparent than real The 
examination in Scotland is really competitive, even if it be less so 
than in England. There is a mark below which the candidate 
does not get a bursary; a mark below which he is not recommended 
to go to a senior secondary school but rather to a junior secondary 
School; a mark below which he is not allowed to pass on to any 
kind of secondary school. "Тһе difference between England and 
Scotland is rather that the competition in Scotland is at a different 
level of ability; but it is still there. 

Not by any means in every Area in Scotland is the examination 
held every six months; and even where it is, one of the examina- 
tions usually has many more candidates than the other. And 
although there may be another examination only six months ahead, 
a boy who has been recommended to go to a junior secondary 
school may take the advice, and not try again. Indeed he is not 


always allowed to do so. 
161 6 
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I feel strongly then that age allowances should be given, at least 


А Мек : A f 
over the six or twelve months within which the main body 0 


candidates were born, and a reasonable period beyond that with 
certain precautions. If this is not done, then the younger children 
born within that period of six or twelve months will win шен 
or will get into secondary schools, in smaller numbers than t a 
older ones, and this is obviously unfair. If it be replied that these 
young candidates have another chance next year, or even in 81 
months' time, I have already answered that in part; and i E. 
case failure is discouraging. But even if they try again, they ү 
still at a disadvantage if по age allowances are given; for аш 
the group of children who are sitting a second time, these are $ 
the youngest. he 
I admit that it is very difficult in Scotland to calculate t Я 
correct age allowance because of the extremely heterogeneor 
nature, in age, of the group of candidates, and the unduly W! E: 
age-range. But these difficulties may be diminished—they ? 
much less in some counties—and are not insuperable. & 
As long as examinations are held only annually, or in some с 
every six months, it must be right to give ап age allowance to 114 
a child born in June have the same chance as one born in Jano 
even if this apparently increases the number of misfits in T 
secondary school. Nor is the reason why it does so far to 566 2 
it is probably because the secondary schools do not give an 28 
allowance over the twelve months in judging a child. The 
agree with the entrance marks without age allowance becaus 
they themselves give no age allowance. Both procedures аге 
unfair, and to that extent are іп agreement. Age allowances are 
sometimes, by those opposed to them, called a premium on youth 
They are not that. When scientifically applied they are a device 
to compensate for the unfair premium on age. 


D 


CHAPTER XVI 


FINAL ORDER OF PREDICTIVE VALUE OF THE 
VARIOUS TESTS AND BATTERIES 


Tue EvipENCE 


To the original follow-up correlations for senior and junior 
secondary schools we have now added the following evidence: 


Best weighted correlations. 
Correlations with the best battery. 
Number of misfits for each battery for the senior secondary 
group. 

For ease of comparison we transformed all the results into 
comparable correlations by methods already explained. 


TABLE XLIII 


Collected correlations with success for the various tests and batteries 


Г---- 
Senior secondary | Junior secondary 3 
Dum schools schools coe ERE 
Battery tion Misit with e E 
in 
minutes| Equal Best Equal | Best battery (6) 
n weighted weighted weghted | weighted 
9) G) (3) (4) 6 |өӨ|60)|%(|0 
| 
1Q+Q+Ts .| .. -8o4 | +808 | -806 | -808 |.8o2| .. | +804 | rt 
1Q+Q+S+Ts| ., +800 +808 +799 Зор | -Зо2 | -799 | 800 | 1-10 
ASETE] - [me | ame | T | 783 1793 72 | О 
+Q.  .| aso | 286 | 7% | -774 | 772 | 793 | 784 | 784 | об 
Ч +Ts A ы 783 787 .78о о | 780 | +777 | 781 | 1-05 
Q+Q+S . 325 | 283 | 2796 | 773 | 780 | 779 | +782 | 2778 | 04 
d *8. .| 270 |o | 7E | 0753 | -756 |277 |275 | 2768 | гог 
I э. AAS - 7738 e |5760 | +762 | +756 | -99 
1045.  .| 330 | 736 | 736 | 78 | 5274 |2736 | 749 | -738 |205 
Е 75 | :698 Е "705 Е -701 | +715 | zer | 587 
Io 55 -691 . -689 -69o | -боо | -690 | -85 
Fictitious batteries 
IQ+Ts 5 : 3 4 -787 | > -787 | 1-06 
s М 779 779 795 797 787 | 794 287: 
IQ+S+Ts . Н +782 +785 :793 -802 | -782 | +787 | -786 | 1-06 
X +T, б B 764 1765 775 777 | 779 | 768 7700 5552 
р . . 2 +720 m 741 m 763 | :724 | 7741 | *95 
~ 
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We have labelled certain batteries ‘fictitious’ for the following 
reason. Teachers’ estimates should not be used unless they аге 


1а+а+тз 
їа+а+5+тз 


а+5+та 


za Za ; Z 

шш E пш 55 uo 
z ек iz ut Е ш сш 
O ЖА 5 ес EE шы ег 
Е o8 5 99 zü Sz 05 
4 > ош ош t o9 оо 
2 & ;® ; оз 26 т] gu 
g g 
с| Е [S3 8 ӛз ша Өз ӱз 
[4 Б: E H НЕ} с, [n Ге 
Gl a £z 5 59 | Ба 29 1 Su 
[5] a ow o эш ош өш әш 


Fie. 42.—Graphical presentation of complete data as to order of merit of the 
various tests and batteries, 


properly scaled on a uniform examination. They therefore imply 
that the results of a uniform examination are available, and would 
be used. We need, in fact, retain no battery which contains Ts 
unless it also contains Q. 
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In view of the results given in Chapter X it is unlikely that best 
weighting will be used in practice; and correlation with the best 
battery cannot be regarded as an authoritative test. We have 
therefore treated these merely as supplementary evidence, and 
used the averages of columns (2), (4) and (6) to give the final order 
of predictive value in column (8). 

‚ The final judgment is best made from the graphs in fig. 42, 
in which the determining measures are shown at the left and the 
supplementary ones at the right. 


CONCLUSION 


Fig. 42, the product of an immense amount of labour, is 
worthy of careful study. Оп all the determining tests IQ 4 Q4 Ts 
Comes out as the best battery, though its superiority over 10-0 
+8+Ts is slight. The difference may, indeed, be due to the fact 
that, in the complete battery, scholastic attainment is over- 
weighted in relation to IQ, an idea which receives support from 
the best weighted correlations. { 

After a distinct gap there is a set of four batteries between which 
the intervals are small. Yet the order is not in doubt. On all 
the determining criteria Q +S + Ts is the best of the four. 10-0, 
while somewhat lower in the correlation for junior secondary 
schools, is decidedly better than Q+Ts, which in turn is better 
than IQ +Q +5. 

"Thereafter the order is perfectly clear. 

4 Turning now to the supplementary data on the right-hand 
Side we find nothing to cast doubt upon our final order, and 
much to confirm it. ‘The only discrepancy is the rise in standing 
of IQ +О +5, due, in all probability, to the fact that scholastic 
attainment is over-weighted. 

'The order and spacing in fig. 42 conform closely to those 
in column (8) of Table XLIII, and we may take the latter as 
d basis for the final graphical presentation of the order in 

Б. 43. 

As far as the order of merit of the single measures is concerned, 
our finding is that a carefully set and corrected examination is the 
best basis for the prediction of a pupil’s success іп the secondary 
school. "Thereafter the order is, teachers" scaled marks, scholastic 
tests and intelligence quotient. 


We have now attained one of our main objects, which was to 
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Tl i combina- 
determine the order of predictive value of the oe T 
tions of estimates, examinations and tests as actually 


RY 
CORRELATION BATTERY CORRELATION 
"804 


largess rs 


а+5+т5 
19+ 


pam 


дуз 


Q 


Fic. 43. 


between the final 
atistically Significant, 


| 
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If the difference between the z values! for апу two batteries is 
greater than “04, the difference between the corresponding correlations 
15 statistically significant? 


The fact that not all the gaps are statistically significant is not 
surprising, and is no reflection upon the precision of ‘our results. 
The more we increase the number of steps within a restricted 
range the more difficult it becomes to establish the significance 
of any one; and it must be remembered that while statistical 
significance establishes a real difference, its absence does not prove 
that there is no difference. 

A large proportion of the mediate differences are significant. 
If, for example, an Education Committee is considering whether 
to use IQ +Q or IQ +5, it will find from the Table that the differ- 
ence of z values is “ІІ, which is nearly three times the minimum 
for statistical significance. It is therefore left in no doubt that 
IO +Q is the better battery. 

_We believe that the graphs in fig. 42 will carry greater con- 
viction to the general reader than these statistical considerations. 
His confidence in the soundness of our final order will be deter- 
mined by the fact that we have three different determinations, 
two absolutely independent and one partly independent, and that 
there are only two small deviations from the absolute conformity 
of all three orders. 


1 Given in column (9) of Table XLIII. 

2 If we have estimates of ғоғ for two samples of n, and л, from equally 
correlated populations we can combine them by the ж transformation. The 
combined result will have the same accuracy as a single correlation obtained 
from n, -- 1, — 3 pairs. 

In our investigation we combined correlations such as гор, Tor, and rary 
but these were determined on the same pupils. We cannot therefore maintain 
that the result is equivalent to a single correlation based on #4 +1, +ñ — 6 pairs. 
In calculating standard errors we counted these as different estimates of the 
correlation for the same sample. ^ 

In this way we obtained the standard error of the correlations for the senior 
and junior secondary schools. The correlations corresponding to the misfits 
are not wholly independent of those for the senior secondary schools, for they 
are based on the same group of pupils. They were, however, obtained by a 
different method, and we counted them as having the same standard error as 
the correlations for the senior secondary group. Knowing the three standard 
errors we then calculated the standard errors of the correlations in column (7). 

The use of ж values has the great advantage that the standard error of z is 
not dependent on the value of the correlation for the total population. 
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SUITABILITY OF THE VARIOUS BATTERIES FOR CONSIDERATION OF 
Вокркк-1лхЕ Cases: Errect ох Окркк or PnrpiCTIVE VALUE 


We have seen that by the aid of certain additional principles in 
borderline cases we can reduce the number of misfits for the 
complete battery by 6. Now, if we use IO only we cannot adopt 
any of these additional principles; with a pair like IQ +Q, only a 
limited number. "This consideration must therefore be taken into 
account in fixing the final order. 

For the senior secondary group, omitting all pupils with absences 
in any item, we found the reduction in the number of misfits for 
all the batteries when the additional principles were applied. In 
so doing we introduced certain special rules to meet different cases. 
For instance, with a battery like О + Ts, Tes and Tas were never 
counted as flops, but an examination mark was omitted if it fell 
below the corresponding teacher's scaled estimate by more than 
77 sigma. . 

The reduction in the number of misfits by the use of additional 
principles in border-line cases is as follows: 


Number of tests in Average reduction in number 
battery of misfits 
4 го 
1 2:7 
= 4 
I оо 


The reduction is greater with the batteries which include the 
larger number of tests, and a study of the complete data shows 
that the only effect on the order of predictive value would be to 
raise IO +Q «S a little in the scale. Apart from that, the result 
would be much the same as that obtained by stretching the line in 
fig. 43, keeping the lower end fixed. 

Тһе detailed figures also suggest that additional principles 
should be used only with batteries of more than two tests. 


RECOMMENDATIONS 
We recommend: 


That all pupils at the Qualifying stage be tested, not merely 
the applicants for bursaries or Ёл admission to senior 
secondary schools, 
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That the battery used in urban areas be I +Q + Ts. 
That, in rural and other areas where satisfactory scaling ot 
teachers’ marks is impossible, the battery used be I +Q. 


PossIBILITY or Escape FROM EXTERNAL EXAMINATIONS 


Our problem was to find the batteries of existing measures 
which are most effective in selecting pupils for secondary educa- 
tion. As scientists we should have had no feelings in regard to the 
answer; but, as educationists, alive to the evils of external examina- 
tions, our satisfaction at the consistency with which our experi- 
ments revealed the best batteries was not unmixed with regret 
that these all contained the Qualifying examination. Having 
stated our conclusions in a dispassionate scientific way, we may 
now give rein to our educational prejudices and consider whether 
there is any way of escape. А 

Perhaps the most promising suggestion is that we should use 
an intelligence test plus teachers’ estimates scaled on its results. 
We thus almost completely avoid the evils of external examina- 
tions, but this advantage would be purchased at a cost. — 

In Chapter V we have given our reasons for believing that 
scaling upon intelligence score is unsound, of which the most 
important are: 


With this method of scaling, the marks for all pupils from a 
particular school might be in error by as much as I2 per 


cent. - 
Since we should scale English and Arithmetic separately 


they would both have to be scaled upon intelligence score, 
and we know that the correlation between school means 
in English and Arithmetic is too low to justify this 
procedure. 

The correlation with success of the best battery is +804, and 
from fig. 43 it is clear that the correlation for the one now 
suggested will be less than 4-787. Keeping in mind all the a 
already given on this point, we estimate that the number o 
misfits in our secondary group of 416 (56 for the best battery) 
would be increased by about 5. 4, 

"That is the debit side of the account; but against it we BS 
the educational advantages to the whole 3,000 pupils in the Quali m 
ing group resulting from the removal of the incubus of externa 


examinations from the primary schools. A 
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Science is not yet capable of putting the credit side iato 
statistical terms, and the decision must be left to the kae 
the philosopher. But the problem is well worthy of cola of 
investigation, particularly in the direction of devising mean 


reducing the debit. This might be effected by such methods as 
the following: 


Improvement of teachers’ standards of marking, for example, 
by encouraging them to make more use of standardised be 

Replacement of the present intelligence tests by some kin E 
composite test, which, while including English and Ari 
metic, would be free from the evils of external lae uec e к 

Increase in the flexibility of the secondary system in the " 
year in such a way that transferences could be easily made an 
mistakes in selection rectified. 


CHAPTER XVII 


PROBATIONARY PERIOD IN THE SECONDARY 
SCHOOL 


Tue following results have some relevance to the last suggestion 
made in the preceding chapter. 


Рвкгістіуе Уліле or Marks ім First Year or SENIOR 
SECONDARY COURSE 


Our method is to compare the prediction of the pupil’s later 
Success made at the Qualifying stage with that which we could 
make during and at the end of the first secondary year. The 
correlations between marks in the first secondary year and later 
Success for the senior secondary group are as follows: 


Correlation with average class marks 


Average marks in ; 
in second year 


Ist term of Ist уеаг . +854 
2nd term of ist year . 4:886 
Ist year . А è 4:894 


These data show the extent to which our predictions would 
improve throughout the first secondary year, and give us a 
statistical basis for considering the best length of the probationary 
period, 


Comparison WITH PREDICTION MADE AT QUALIFYING STAGE 


For this purpose we take the results for the best battery, 
IQ+Q+Ts, and we must use the second-year average marks, 
for which, as we have seen, the correlations were rather low. 
This ensures that the ranges are the same for the correlations 
which we are comparing. Making also the allowance for the 
reductions in number of misfits by the use of additional 
principles in border-line cases (translated into correlation terms 
by the methods of Chapter XIV), we find that the comparable 


correlation is 
7ае+о+те Fay = +°829 
171 
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Тһе position is therefore that with a good battery at the Qualify- 
ing stage we can reach a correlation of -829; at the en 
term of the first secondary year the figure is only “02 
Тһе difference is not so decisive as might have been expecta 
when we consider how heavily the dice are loaded in pt. 
the later prediction. The decision is reached six months өс A 
by which time the pupil has reacted to the new ipee $ 
the secondary school; it is based on a much larger num ш. 
tests, including those in the new subjects of the secondary dB 
culum. "These are legitimate advantages of the prediction m: le 
at the end of the first secondary term, but there 15 а possi 
illegitimate one which arises from the halo effect. 


5 р AT 
Maximum CORRELATION WITH Success TO BE ЕХРЕСТЕ 


ТНЕ QUALIFYING STAGE 


One very interesting deduction which we can 
above results is that no improvement of testing a Jation 
stage is ever likely to produce a battery giving а higher corre cou 
than -854; for we could hardly expect that such a battery tua. 
yield a better result than a full set of examinations 1n the ac 
subjects and in the environment of the secondary school. 


s р : s e of 

We feel, indeed, that we can, without incurring а chains 

undue complacency, hold that our correlation for E end 
t 


must be very near to the high-water mark of prediction а 
of the primary school course. 


Minimum NUMBER or MISFITS то BE EXPECTED WITH ANY 
SYSTEM or SELECTION Ат THE QUALIFYING STAGE 


By the methods described in Chapter XIV we find that if the 
number of misfits for our best battery were 53, then: 


If the prediction were made on the basis of the first-term marks 
in the secondary course, this would be decreased by 4. 

If made at the end of the second term the decrease would be 8. 

If made at the end of the first year the decrease would be 9- 


We can also prepare the following Table showing what the 
complete results of selection would have been if made at the en 
of the first term in the senior secondary school. It is applicable 
to conditions, likely to obtain throughout Scotland, where the 
distribution of applicants is skew and the proportion of unfit 


v 
! 


SO 
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candidates is 25 to 30 per cent. We have therefore given the 
figures in percentage form: 


Successes Failures Total 


Admit 22 66:9 6:9 738 
Reject “ 3°9 22:3 26:2 
Total . 708 : 29:2 100-0 


These figures are very important, in that they represent the 
best result that could possibly be obtained by any system of 
selection at the Qualifying stage; and they point to the following 
practical conclusions: 


It is most unlikely that, under the conditions stated above, any 
system of selection at the Qualifying stage will ever yield 
results where: 


(1) The number of misfits will be less than 10-8 per cent. 
of the applicants for admission to the senior 
secondary courses. 

(2) Less than 9:3 per cent. of the number admitted will 


^ fail to make good.! 


(3) Less than 4:9 per cent. of the applicants will be 
erroneously refused admission. 


APPLICATION oF Fonrcorsc RESULTS TO THE PROBLEM OF 
THE PROBATIONARY PERIOD 


Let us consider first the suggestion that the probationary 
period should be one year. We should then have 9 fewer misfits 
(44. instead of 53) when the final selection of course was made— 
a very considerable saving. But, as usual, there is a debit side to 
the account. If, in the probationary year, we do not introduce 
the new characteristic subjects of the senior secondary course, 
we have no real test of the child's reactions to these subjects, and 
the prognostic value of the probationary year would be con- 
Siderably reduced. If we do introduce them, we have to set 
against the saving of 9 misfits the serious wastage resulting from 
teaching these new subjects to a whole group, many of whom 
will give them up at the end of one year. 

An exact assessment of this wastage is beyond our scientific 
power, but we would emphasise one point and make one suggestion. 


1 Assuming, of course, that the correct pass-mark has been used. 
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Whatever procedure be adopted in regard to a probationary peus 
we cannot escape the necessity for making as careful а oen. s 
possible at the Qualifying stage; for we must then decide whic! ш 
our Qualifying group are to get the chance of taking it. То 845 
the chance to all would be sheer educational folly. If a care P 
selection for the probationary year is made, we suggest that : 
compromise procedure should be adopted in regard to the conte 
of the course, giving only one foreign language. Thereafter, ге 
that we сап do is to try to ensure that change of course, or eve 
of type of school, is as easy as possible. Д ба 
Probationary periods of less than а year have been tried. A 

this point our results indicate that the superiority of решен 
after опе term over prediction at the Qualifying stage 15 hardly 
sufficient to warrant the dislocation of organisation which such an 
arrangement would cause. A period of two terms, while giving 
à better basis of prediction, is unlikely to be considered. 


CHAPTER ХУШ 
THE MISFITS 


Arter trying all possible methods, and using all the supplementary 
data at our disposal in regard to health, industry and personal 
qualities, we were still left with 5r utterly refractory misfits.* 
"There were 32 pupils who were failures in the senior secondary 
course, and, no matter how we used our data, they were admits; 
and there were 19 pupils who were rejects at the Qualifying stage, 
but who turned out to be successes. The time has now come to 
examine these with a view to finding pointers to the next steps 
in the improvement of our selection procedure. 


Tue Арм:їт-ЁАп5 


EXTENT TO WHICH THESE WERE BORDER-LINE CASES 


One would expect that the majority of these final obstinate 
misfits would be pupils close to one or both of the admit-reject 
and success-fail border-lines; but an examination of their Qualify- 
ing scores shows that this is not so: 


Qualifying score Number who were 
(IO +0 +5 + Ts) Boys Girls Total doubtful failures 
(sigma) in secondary course 
+1-40-1-49 T I I I 
+1:30-1:39 2 oe 2 53 
--I:20-1:29 4 3 7 4 
+1-10-1-19 2 .. 2 an 
+1:00-1:09 4 I 5 2 
+ +90— :99 3 3 6 3 
+ -80- -89 4 2 6 ya 
% 99-549 2 I 3 
"Total 21 II 32 10 


"There is certainly по evidence of bunching at the admit-reject 
line; and if we regard as border-line cases those falling between 
4-6 sigma and 4-8 sigma, only three of the pupils would be 
counted border-liners on the entrance test. "Тһе last column 

1 This was the number for whom we had absolutely complete data. 
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indicates the number of cases, Io in all, where the Бае 
gave his judgment as to success in the secondary “> oa 
hesitation: all the others were perfectly clear ве i 
therefore take it that only 13 out of the 32 admit-fails fa 

category of border-line cases. 


ILLUSTRATIVE CASES 


e = қ clear admit 
The girl whose profile is given in fig. 44 isa p a 
on all tests; the home conditions are quite good, an 


+3 


+2 


+1 


Fic. 44.—The admit-fail with the highest Qualifying score: Failure due to health. 
primary nor the secondary head master has any fault to find with 


- She herself was not particularly keen to 


take a senior secondary Course; but she had no strong objection 
to meeting her parents? wishes. 


о be a case where the girl has been carrying 
on a losing struggle against indifferent health. At the Qualifying 
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аз a general principle such a procedure would increase the total 
number of misfits. The only suggestion for improved selection 
arising from our consideration of this case is that where there is 
ground for serious doubt as to health the pupil concerned should 
be specially examined by the School Medical Officer. 


We cannot see that we could have been blamed for admitting 
the boy whose profile is shown in fig. 45. There is not a single 
unfavourable indication in all his Qualifying data. His entry to 
the secondary school course seems to have been determined by the 
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Fic. 45.—A failure due to home conditions. 


fact that he was advised by his primary head teacher to sit 
the bursary examination. He was successful; and one gets the 
impression that it was a Scottish reluctance to give up the bursary 
rather than any keenness for secondary education that decided 
him to avail himself of it. 

The view of the secondary head teacher was that the difficulty 
lay in thehome. "Тһе boy got no encouragement from his parents, 
Who seem to have soon regretted their decision to enrol him for 
the five-year course. As a consequence he had little interest in 
his studies and a strong desire to get away to work. One can well 
understand that this lack of home encouragement, while not 
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-affecting the primary record, might have a more serious influence 
when the boy approached the leaving age. 


In the puzzling case shown in fig. 46 something must have 
happened in the girl's history at the beginning of the second year 
in the secondary school. She is a perfectly clear admit on the test 
results, is above average in practical ability and in health, is graded 
A in industry; and the primary teacher thinks that she will be not 
merely a clear but a distinguished success in the senior secondary 


IQ Se E Tes Qa 


Fic. 46.—A failure due to emotional factors. 


school. The primary report mentions perseverance and self- 
reliance as her marked personal qualities. The home conditions 
were favourable on the whole, but she was “towed жо take life 
too easily.’ 
How then can we explain this remarkable drop to the level of a 
sure failure at the end of her first year? ‘The only answer we can 
give is speculative. In discussing the pupil with the head master 
we were told that ше drop was accompanied by signs of a definite 
change in personality, the details of which might pj lue to 
identification. The child was also ight give a clu 


c handicapped by a certain 
deformity, perhaps not of much import at the шау stage, the 
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dawning sensitiveness to which might have serious effects on the- 
temperament and character of an adolescent girl. 

Perhaps the main pointer here is that such pupils should be 
referred to a Child Guidance Clinic. At any rate, no improve- 
ment in tests and examinations will save us from wrong placements 
in cases of this kind. 


Fig. 47 gives the profile of a boy who is just above the admission 
mark but is a very clear failure in the secondary school. Two facts 
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Fic. 47.— The admit-fail with the lowest Qualifying score. 


seem to have contributed to his lack of success. In the first place, 
he was ‘rather indifferent to school work’ in the primary classes, 
his interests being mainly in games. He was not consulted in 
regard to his enrolment in a senior secondary course, went there 
with a grudge, objected to ‘too much home work,’ and left to 
become a salesman. "Тһе second cause was no doubt his weakness 
in Mathematics. Asked in Miss Rodgers’s questionnaire} as to 
the subjects which he found difficult, he mentioned Mathematics, 
adding, ‘It all depends on the teacher. . . He must have had 
something else of interest to add here, for he had started another 


1 See p. 198. 


р ION 
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i is profile shows 
clause, but unfortunately thought better of it. Ж... pro S 
that he is very good in English, but he just po Mieres Со 
of the minimum of o sigma in Arithmetic. py 
overcome such a weakness, but the mediocre or pem oon aed 
continued failure and discouragement in one subj 
all the others. 


Tue Reyecr-SuccessEs 
-LINE CASES 
EXTENT TO WHICH THESE WERE BORDER-LIN 


4 as follows: 
The Qualifying scores of the reject-successes were 


Qualifying score . pecu куы ы 
ао Ў Q E x Ts) Boys Girls Total a E 
sig: х 
+:90—:99 T I I 
+-80—89 ae 
+-+70—79 1 44 ї e 
+-60--69 2 1 3 : 
*'50-59 2 2 4 : 
+:40—49 3 ї 4 : 
%:309-:39 І Т 2 5 
+-20—29 2 В 3 : 
%:10-19 I xe I 
~ Total 12 7 19 9 
1 : they 
The two pupils above +70 sigma were rejects because 
fell below the min 


deg bero 
imum of О sigma in IQ. The total num 
border-line cases is 13. 


ILLUSTRATIVE CASES 


We now know that the pupil whose profile is shown in fig. 48 is 
a boy of intelli 


ted him on his Qualifying data. The primary 
teacher indicated self- 
after reading his repli: 
inclined to agree. H 
doubt because financial 
curred in his parents? vi 


„А0 
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Ind H Fc. I I Ш 5 
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Fic. 48.—A border-line reject but а clear success. 
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Fic. 49.—A border-line reject not much above the success line. 
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well-educated to find decent work? Choosing the F. Шаа 
course in preference то F rench-Latin because he ind a ЕЗ 
language would be more useful in the present day, r дус E 
no reason to regret his choice, The main reason for " su р. 
the opinion of the Secondary head master lay in dn 
parents gave him every encouragement and were keen Жам iie 
in his progress. We are glad that our rejection of ши a 
merely fictitious, and that he has had, and taken, his chance. 


d 
Fig. 49 illustrates the type of case which we should hane € 
to be much more common among those irreducible mis we Бат 
boy is not far below the line for admission; indeed, if he дің разз- 
just a little better in English he would have earned Meis s келе 
mark for special ability in one subject. He із also very litt dest. 
the success line, and the head teacher gave his decision wit 
hesitation. " ie 
The boy’s ambition is to become a doctor, but he is B 
emphatic on the point that he is not in the right course. ага 
languages, he says, “аге too much for me? We hope ће уу 


t. 
his fight which, but for good home conditions and encouragement, 
he would probably have lost. 


Fig. 50 gives the profile of a bespectacled little boy from aE 
home who did not sit the bursary examination. In the primary 
school his interest in schoo 
of Music, was about average, 
the admission line, The ch 


his own, though one gathers that he did take some account of his 
parents’ wishes; an, he is quite satisfied that it was a wise one. 
At first we Suspected that 
end of the first year might Ь 
head teacher was of opinion th. 
personality rather than of intellect, 
of the second year the boy s 
responsibility and then bega. 
had never done before, 


The ty 
follow-up е Oy is a cl 
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Fic. 50.—A possible ‘late developer.’ 


19 ае E TesQa A Tas Av. Тр Ind.H Fc. I II II 5 


Fic. 51.—A pupil who repeated the first-year course. 
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following year. "Taking account of all the facts, the май sadi 
was of opinion that he would stand the pace throughout the пос F 
and obtain the Leaving Certificate within the six years. 


i interested 
steady persevering boy who comes from a good and inter 
home. 


Awatysis or Causes or DISCREPANCY 


5 то 
PROPORTION OF CASES WHERE DISCREPANCY MAY BE e 
UNRELIABILITY OF ADMISSION AND SUCCESS MEASURE: 


Misfits may be due to lack of reliability of the mei 
pass-marks for admission and success, or to extraneous factor: Ea 
failure of health, deterioration of home conditions, etc. да vill 
portant to find out the relative proportions of these, for t p nE 
be our main pointer as to directions of further advance. us; df 
former number is large we must strive to improve our tests; 


the latter, we shall have fuller knowledge of the limits of any 
predictive procedure. 


Р y A Я Total 
Possible cause of discrepancy  Admit-fails Reject-successes 
Unreliability of measures 
Unreliability of entrance tests 10 


14 
4 
Unreliability of success grading 3 9 d Е 
13 13 26 
Extraneous factors (such as health) 13 5 P: 
No satisfactory explanation 6 1 4 
dota Е : 32 19 ЗЕ 


еге the success grading might be wrong; the 
pupil of fig. 47 as one where the 


fault. Further, unless we уу 


І ез We have no satisfactory reason. 
ainly Over-estimates; so we have full con- 
fidence in concluding that in aż least one-third of the cases the 
discrepancy is due to extraneous fac 


ә tors and could not be avoided 
by any improvement of the tests and examinations, 


bans 
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EXTRANEOUS FACTORS CAUSING DISCREPANCIES 


We selected in each of the 18 cases what we thought to be the 
main extraneous factor causing discrepancy and found the position 
to be as follows: 


Cause of ашшы LE animis: Total 
discrepancy Boys Girls Total Boys Girls Total 
Home conditions 3 I 4 1 % 3 1 
Desire to take up 
work 2 I 3 < 3 
Health 2 zs 2 т 1 3 
Unsuitable course I I 2 р É 
Personal qualities I I ы $ 5 
Changes of school І І d i 
Total . 10 3 13 2 3 5 m 


In any one case, however, there are usually several contributory 
Causes, and we present below the number of times that the various 
factors were mentioned in the reports. This gives a fairer measure 
of their relative importance: 


Number of times mentioned 
Factor 


Admit-falls | Reject-successes Total 
Home conditions š 12 10 22 
Unsuitable course — . 10 z 19 
Health . А n 6 д 
Personal qualities " 6 I 7 
Desire to take up work 4 us ij 
Interest in a particular 
subject . Ў 5 es 4 4 
Changes of school » 2 2 
Total . 5 4o 17 57 


It will, of course, be understood that by health and personal 
qualities we refer to changes which took place after the Qualifying 
stage. The category, ‘interest in a particular subject, may be 
illustrated by a girl of indifferent Qualifying attainments who was 
talented in Árt and desired to become an Art teacher. She realised 
that a successful secondary course was a necessary part of the road 
to her goal, and this provided an incentive which pulled up her 
work in all subjects, 
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SUGGESTIONS FOR [MPROVEMENTS ім МЕтнор$ or SELECTION 


The smallness of our numbers, added to a consciousness of the 
deficiencies of our technique, precludes any definite Peu 
: В . т 

Yet our experience in working with these cases for nearly six yea 


suggests the following directions of improvement in methods of 
selection. 


Reports from the Primary Schools 


We assume that the primary teachers will be encouraged 55 
shoulder more and more of the responsibility for selection an 
guidance; and that, by a better technique than ours, they will Р 
invited to give reports on home conditions, health, interests MES 
personal qualities, Іп our view it would be both unnecessary an 
undesirable to ask them to make such reports on every applicant 
for admission to a senior secondary course. It is much better E: 
have full and careful reports on a few selected cases than eg" 
терогіѕ on all. The knowledge that these reports would be 
carefully considered would increase their reliability. 


Special Medical Examination in Doubtful Cases 


If the primary teacher expressed doubts as to the child's physical 
fitness to face the strain of a senior secondary course, the pup! 


should be referred for special examination to the School Medical 
Officer. 


Enlistment of tbe Resources of the Child Guidance Clinic 


If the primary teacher was of opinion that the child’s success 
might be prejudiced by the home conditions, these could be 
reported on by the Social Worker of the Clinic. Cases of difficulty 
in regard to personal qualities or cases of doubtful IQ could with 
advantage be referred to a Clinic Psychologist, who should also 


be consulted when a child suddenly changes in personality or in 
class standing during the secondary course, 


on Use of the Flexibility of the Regulations for Senior Secondary 
ourses 


The number of misfits arisin through itabili f the 
course will be reduced if the Eds Vc ru qe i A 


red ucation Committee uses a good 
battery of Qualifying tests and sees that the results are sent to the 
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secondary head teachers: but certain changes within the secondary 
school are also needed. In particular, there might be a reduction 
in the number of pupils who are encouraged to take two foreign 
languages, a greater provision of courses for adolescents whose 
cultural tastes are not met by the old ‘normal general course,’ and 
a greater effort to exploit the incentive that comes from special 
talent in subjects like Art and Music. 


ExrRANEous FACTORS AFFECT Boys MORE THAN GIRLS 


One curious feature of our results is that in the misfits the boys 
always outnumber the girls, particularly in the admit-fail category. 
Here the ratio of boys to girls is 21 : ІІ for the Senior Leaving 
Certificate, and 19 : 8 for the Junior Leaving Certificate. This 
Seems to indicate that a girl tends to work up to her capacity 
better than a boy. It is possible, too, that a girl is not so 
liable to the discontent and unrest that come over so many boys 
in adolescence and lead to a desire to get away from school at 
all costs, : 


CHAPTER XIX 
: BURSARY EXAMINATIONS 


Tue object of bursaries for senior secondary courses is to ensure 
that no child who shows promise of profiting by such an education 
will be debarred therefrom by reason of the cost; and in the em 
where the Inquiry was conducted awards were, in 1935-1939» 
made on the basis of a special bursary examination. ‘The numbers 
of candidates and of bursaries awarded were: 


Group Candidates Bursaries awarded 
December 1935 140 53 
June 1936 . 275 130 


Is А SpecraL Bursary EXAMINATION NECESSARY? 


It has been a fairly common practice in Scotland to set a special 
examination on a higher standard for the award of bursaries- 
"This was no doubt based partly on the belief that a stiffer examina- 
tion would give a wider scatter of the good pupils, and ensure that, 
at the pass-mark, fewer candidates would have the same score. 
Clearly, if the examination were so easy that the clever puprs 
all scored full marks, this extreme ceiling effect would make the 
results useless for bursary purposes. 


THE CEILING EFFECT 


So far as we can see, the fear of a ceiling effect is no sound reason 
for a special bursary examination. In the first place, it is quite 
possible to have an examination for a complete Qualifying grouP 
which has neither a ceiling nor a floor effect. Our own examina- 
tion showed neither; and reference to fig. 561 will show that 
this holds also for the distribution of scores оп the complete 
battery. In the second place, the ceiling effect would be no dis- 
advantage if the pass-mark fell some way below the point where 
the frequency was greatest. 

1 P. 197. 
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SPREAD OF CANDIDATES AT THE PASS-MARK 


'This consideration is certainly important; but it is rash to 
assume that a stiffer examination will necessarily effect an im- 
provement; it may have the opposite effect. < 

While the fact that the bursary histogram in fig. 52 is lower on 
the scale is not conclusive proof that the examination was stiffer, a 
study of the papers confirms the belief. It will be noted that the 
frequencies at the verticals which cut off the actual number of 


—— — — BURSARY EXAMINATION 


— QUALIFYING EXAMINATION 


FREQUENCY 


SCORE 


Fic. 52.—Score-scatter of December bursary candidates on the bursary examination 
and on the Qualifying examination. 


successful candidates are very nearly the same in both examinations. 
The stiffer examination has therefore failed to secure a better spread 
at the critical point. 

Fig. 52 shows how much of the effectiveness of the stiffer 
examination in spreading the candidates at the pass-mark depends 
upon the point at which the pass-mark is drawn. If it had been 
necessary to select less than 6 candidates, instead of 53, the stiffer 
examination would have been the better; but for almost any 
other number the easier examination would have been as good a 
basis of selection. Now, in Scotland it is never likely that only 
6 candidates out of 140 would be entitled to bursaries; and we 
may therefore conclude that no statistical advantage is gained by 
a special examination of the standard used in December 1935. 
Те is, of course, possible that a stiffer examination may give a 
better order of merit, though we think that this is unlikely. 
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ARGUMENTS AGAINST THE SPECIAL Bursary EXAMINATION 


Our view is that a special examination is neither necessary Ж. 
desirable; and that bursaries should be awarded on the basis o: 
the Qualifying scores obtained by the application of a good battery 
to the whole Qualifying group. In arriving at this conclusion we 
had the following considerations in mind: 


Qualifying scores for the complete group are required, not only 
for the award of bursaries, but also 
(а) For guidance and selection for post-primary COUTSES. 4 
(P) As a basis for classifying entrants in the senior am 
junior secondary schools. 

If teachers? estimates are to be used, as is very desirable, they 
must be scaled; and the scaling is possible only on the basis 
of an examination of the complete group. A 

Without Qualifying scores for the complete group there is no 
satisfactory means of fixing the pass-mark either for bursaries 
ог for entrance to senior secondary courses. d 

A single examination is unjustifiable as a basis for the awar 
of bursaries for secondary education. 

Such an examination has no advantage over the Qualifying scores 
either in absence of ceiling or floor effects, or in spread О 
candidates at the pass-mark, 
he institution of a Special bursary examination offends the 
Principle of ‘parsimony of examinations,’ which should have 
the same honour in the eyes of the educationist as * parsimony 
of hypotheses? in the eyes of the scientist. 


ary December June 
examination and (N=140) (N=275) 
IQ +501 +595 
о +-591 +544 


The fact that the correla 
corresponding figures for ou 
smaller standard deviation of the bursary group. 


The most interesting point is that the correlations between 
the bursary examination and our own are as low as -591 and 544. 


tions with IQ are not so high as the 
T own examination is partly due to the 
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AN APPLICATION OF THE GENERAL PRINCIPLES OF CHAPTER XIV 


With a correlation of -591 between the two examinations the 
discrepancies would be very great, as the reader will by now realise. 
If, however, he has read Chapter XIV he should also be able to 
translate this indefinite statement into a fairly exact one; and 
by so doing he will have an opportunity of testing the usefulness 
of the methods therein described. ) 

The following is a summary of the steps. We are selecting 
53 candidates out of 140, so the number of discrepancies for zero 
correlation would be 66. From fig. 30 we see that the number 
for a correlation of -6 would be 62 per cent. of this, namely, 41. 
That is, 20 candidates who get bursaries on our examination 
would be refused awards on the bursary examination; while 20 
Who were refused on our examination would receive awards on 
the bursary examination. The actual number was I9, showing 
that the method gives quite a good estimate. 


Ir an Epucation COMMITTEE HAS ASCERTAINED THE NuMBER or 
SUCCESSES AMONG THE CANDIDATES AWARDED BURSARIES iT 


CAN ESTIMATE THE NUMBER or POTENTIAL SUCCESSES AMONG 
THOSE WHO WERE REFUSED 


If the title of this section had not been so long we should have 
added ‘and it would be most salutary for it to do so? As a rule 
the candidates who are refused bursaries do not go into senior 


Secondary courses, А large proportion of the mistakes made by 
the bursary scheme 


are therefore buried in the junior secondary 
Schools; they do not obtrude themselves on the notice of the 
Committee, and this blind spot in its administrative procedure 
may 


Bive rise to an attitude of unwarranted c 
also gives the Committee little opportunity of knowing whether 
the pass-mark is fixed too high or too low. 

The soundest method is, of course, a follow-up of the complete 
group of candidates; but this involves a considerable amount of 
labour, and a fairly reliable estimate can be obtained by the 
following short-cut. 


After the bursars have been а 
schools the Director of Education 
been successes and how 
December bursary group 
10 failures and 43 successes 


omplacency. It 


year in the senior secondary 
would find out how many have 
many have been failures. Using our 
аз an illustration, the numbers were 
- Thereafter the procedure somewhat 
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resembles that used in the reconstruction of a prehistoric animal 
from a few bones. 

All we know is the pass-mark AB (fig. 53), the numbers 10 and 43; 
and the total number of candidates, namely, 140. Our aim is to an 
an estimate of the number which should go into the quadrant me А 
We must, of course, assume normality of the distributions, an W 
must have an estimate of the correlation between the € 
examination and success. "Тһе latter need not be very exact fo 


BURSARY EXAMINATION 


SUCCESS IN SECONDARY SCHOOL 


Fic. 53.—Estimate of number of successes among candidates refused bursarics. 


this purpose, and the estimate could easily be made by any Director 
of Education who has been using scientific methods in his Area. 
In our own case the estimate would be T7. 

Using the notation of Chapter XIV, we know that 


ГА T7 
k= +530 
СЕВ =30-7 per cent. 


Now, knowing the volume CEB and the position of one of the 
planes which bounds it, namely, EB, it is possible by the method 
indicated in Chapter XIV to find the position of the other, namely, 
CE. We then find that b=oo, from which it follows that the 


number which should go into the quadrant BED is 27. 
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The conclusion is that 27 out of the 87 pupils who were refused 
bursaries would have made good in the senior secondary school. 
The number of correct awards was 43, and the number of 
mistakes 37. It will now be understood why we recommend this 
аз a salutary type of calculation for Education Committees; but 
to allay possible suspicion of the mathematical character of the 
method, we proceed to give the actual numbers, 


Anatysis or Bursary Awarps IN THE LIGHT OF LATER 
Success 


From our follow-up data we know the number of successes and 


failures, both among those who were awarded and those who were 
refused bursaries. 


TWO SAMPLE CASES 


We first give two sample cases in which we compare the predic- 
tions made on the basis of our complete battery and of the special 
bursary examination. Both are from the December group. 


Fic. 54.—A pupil who was awarded a bursary, 
The first, whose profile is shown in 


was thirty-ninth on the bursary list 
award, 


fig. 54, is that of a boy who 
and therefore received an 


7 
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It will be noted that, on our Inquiry results, we should T 
had no hesitation in refusing a bursary to this pupil; and i E 
performance in the senior secondary school shows that our decisio 
would have been correct. < 

For comparison we give, in fig. 55, the profile of a pupil who E 
sixty-ninth on the bursary list and was therefore refused an aware: 


was 


Fic. 55.—A pupil who was refused a bursary. 


This boy, to whom we should willingly have granted a bursary, 
went on to a senior secondary school at his parents’ expense. 


COMPLETE RESULTS 


The fitness of bursary candidates of both December and 225 
groups for senior secondary education is shown below: 


Successes Failures Total 
Awarded bursaries I 28 183 
Refused bursaries 5 163 232 
Total З 224 191 415 


Тһе result is enlightening. Of the 183 candidates awarded 
bursaries 28 were unfit for senior secondary education; r3 of these 
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28 were, in fact, judged to be unfit even for a three-year junior 
secondary course. Even more disturbing is the fact that no less 
than 69 pupils who were fit to profit by a senior secondary educa- 
tion may have been debarred therefrom by reason of the cost. 
Тһе total number of mistakes is 97 in all. 

For the December group the actual number of refuse-successes 
is26. The theoretical estimate in the last section (27) was there- 
fore a close one. 

We trust that the reader has by this time not merely a vague 
suspicion that the pass-mark in these bursary examinations was too 
high, but has proved that this must be во. Since the number of 
Successes is greater than the number of failures, the number of 


award-fails should be greater than the number of refuse-successes 
at the correct pass-mark. 


Tue Pass-Mank ім Bursary Examinations 


We shall assume that the battery 1+Q+Ts is used and that the 
pass-mark should be determined solely on the basis of the fitness 
of the candidates to profit by a senior secondary education, 

We have shown that, on this battery, a candidate for admission 
to the senior secondary schools has an even chance of success if his 
Qualifying score is +70 sigma. It might therefore be expected 
that the same pass-mark should be used for the award of bursaries; 
but this does not follow. Тһе position of the Correct pass-mark 
depends on the nature of the group, its social class, home condi- 
tions, and so оп. We found, for example, that the point at which 
the child in the whole Qualifying group has an even (statistical) 1 
chance of success in the senior secondary course is not +:70 sigma 
but 41:34 sigma. 

Now, for bursary candidates the ex 
more favourable than in the complete 
each case there is evidence of a desir 


* 1:34 sigma. 


€ determined the correct pass-mark (that is, the point at which 
1 See pp. 214-215. 
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half of tbe bursary candidates would be successes) by the methodi 
already explained, and found it to be +-975 sigma. This € 
‘that the pass-mark used in the special bursary espada Td 
at least 6 per cent. too high. We explain in Chapter XX Ae, 
this pass-mark can be used in practice, but it should be Mns 2 
that it holds for the battery ІО--О--Тө. With a poorer bat ry 
the pass-mark would be lower. 


EST 
RESULTS ir BURSARIES HAD BEEN AWARDED ох Basis or B 
BATTERY WITH irs Correcr Pass-Mark 


In awarding these bursaries we should have selected the Бан 
1Q+Q+Ts; and, not having any financial responsibility, ihe 
should have used the correct pass-mark. Had we done 50, 
results would have been as follows: 


Successes Failures Total 
Awarded bursaries 192 41 233 
Refused bursaries 32 150 182 
"Total 224 191 415 


This should be carefully compared with the actual awards EO 
оп p. 194. Short-sighted ratepayers might be inclined to Өү 
at the expense of the 50 additional bursaries that we should hav 
awarded; but, taking a wider social view, we find some com- 
pensation in the facts that we should have admitted 37 more 
pupils who would have been successful in a senior secondary 
course, and that we should have had 24. fewer mistakes. d 

We believe that these figures represent the best result that coul 
be expected in the award of bursaries under present conditions. 


СНАРТЕК ХХ 
SURVEY OF THE QUALIFYING GROUP | 
Кок our survey we use as а basis the average standard scores on 


the complete battery IQ +Q +5 + Ts, which we call the Qualifying 
Scores, 


FREQUENCY 


QUALIFYING SCORE 
(sigma) 
— Distribution of Qualifying scores for complete Qualifying group. 


Fic. 56. 


Comparison or Boys AND Girts 


COMPLETE QUALIFYING GROUP 


There is no significant difference b 


etween boys and girls, either 
11 mean score or in scatter. 


Тһе difference of means is "O17 
sigma ! in favour of the girls, and the standard deviations are 99 
sigma and -96 sigma for boys and girls respectively. 
* "This is about half the standard error of the difference, 
197 
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SENIOR SECONDARY GROUP 


The mean score for the boys is +1-018 sigma and that for the 
girls +1-208 sigma. The difference is more than three times its 
standard error, and is therefore statistically significant. | d 

In an earlier investigation by Mr Robert Robertson it was Sat 
that the girls were better than the boys in intelligence m 
scholastic attainment separately, and that this result e 
not merely for the group as a whole, but in all but one of the 
individual schools. The school which was the exception was the 
smallest. 


JUNIOR SECONDARY GROUP 


In the junior secondary schools the boys are slightly better Low 
the girls, the means being +-167 sigma and +-125 sigma. Te 
difference is, however, not statistically significant. d 

If these tendencies should be confirmed in other years an 
Areas it would be of some interest to consider why the ye. 
secondary courses should attract fewer of the clever boys than 0 
the clever girls. Part of the answer may be that the roads to 
the vocations which attract the clever girl lie through the senior 
secondary school; but it may also be that the drags and pushes 0 
the social environment, whose resultant is directed away from fall 
secondary education, have more effect on boys than on girls. 


Facrors warcn Ретекміхе Снотс or SrcowpARy COURSE 
(In collaboration with Miss Hilda C. Rodgers, М.А.) 


The exacting nature of the main investigation did not permit 
us to make a full.study of this interesting topic, but Miss Rodgers 
obtained some preliminary results from а questionnaire given (0 
391 senior secondary, and 420 junior secondary, pupils. 

mittedly the problem cannot be solved in this way; the children 
do not know all the facts; we get little light on the motives which 


influenced the parents, or on the extent to which poverty debars 
pupils from senior secondary education. 


CHOICE OF TYPE OF SCHOOL: sENIOR SECONDARY OR JUNIOR 
SECONDARY 


One of Miss Rodgers's questions to the 
was, *Why did you take a senior secondary rather than a junior 
secondary course?" and she gave a choice of three answers: 
* (a) Because I was advised to go to a senior secondary school by my 


senior secondary pupils 
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primary head master. (b) Because my parents wished me to go to 
а senior secondary school. (с) Because I myself desired to go to 2 
senior secondary school? She also left a space for ‘Other reasons. 
We thought that this would seldom be used, but it elicited a large 
variety of frank responses, given with a sublime disregard for the 
requirements of exact statistics. For many pupils the choice was 
determined simply by the fact that they had been in the primary 
division of a senior secondary school. As one casual young lady 
put it, “Т was in . . . and just carried оп’ For others, the fact 
that a senior secondary school happened to be the one nearest to 
the child's home was the main consideration; quite a number of 
young educational oracles had the view that a senior secondary 
school gives a ‘better education’; a fair number admitted candidly 
that they went to a senior secondary school simply because their 
friends had gone there; one or two, apparently with snobbish 
Propensities, were most influenced by the fact that they would 
mix with pupils of a better social class. : 

Pupils were asked to indicate the main reason for the choice, 
and we give below what statistics we were able to extract: 


Reason for choice of type Senior secondary | Junior secondary 
of school group group 
Advice from primary school . 28 12 
Pupils desire а ^ 7o 144. 
Parents! desire T 4 r 196 95 
Other reasons " š и 78 121 
Total . - 372 372 


Even if we allow for their unreliability, the figures suggest that 
only to a limited extent is advice from the primary school a deter- 
mining consideration in the child's choice of type of secondary 
school. Parents’ ambitions and aspirations appear to be the main 
factor in the senior secondary group, and the pupils’ own tastes іп 
the junior secondary group. Confirmation of this was given in the 
answers to another of Miss Rodgers's questions: *Were your wishes 
taken into account when your Secondary course was decided 
upon?' "То this question 94 per cent. of the junior secondary 
Pupils gave an affirmative answer; in the senior secondary group 
only 73 per cent. 

f the 121 in the junior secondary group who gave ‘other 
reasons, 59 indicated that they wanted to leave school at the 
earliest possible date, and 20 gave as their reason that they had not 
been awarded bursaries. "Тһе others defied classification. 
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M NDARY 
CHOICE OF COURSE WITHIN SENIOR OR JUNIOR SECON 
SCHOOL 


В ы Әді ich 
Miss Rodgers asked a similar question on this point whic 
brought out the following replies: 


; dary 
Reason for choice of Senior secondary Junior ey 
Course group 8 12 
Advice from primary school. 15 7 
Advice from secondary school . 69 149 
Pupil’s desire : a E 85 98 
Parents? desire = Я 104 73 
Other reasons a м 3 89 
Total . š 360 339 


5 се 
Again we see that the pupils’ own desires have more e 
in the junior secondary group and those of the pe from 
senior secondary group. It is gratifying to find that а i tter. 
the secondary head master is an important factor in the la o less 
"That old traditions die hard is shown by the fact that ШЕ А 
than 3 pupils chose their course because of the ‘value of Latin. 


TRANSFERENCES BETWEEN SENIOR AND JUNIOR SECONDARY 
SCHOOLS 


Our results show that ease of transference from one type e 
school to another, when a mistaken choice has been made, 
always likely to be an essential feature of an educational system. 
It is therefore interesting to find how far advantage is taken © 
such facilities for transfer as the present Scottish system offers. t 

Miss Mitchell collected particulars of every change made by 


pupils in our Qualifying group and found the position to be as 
follows: 


Not a single senior secondary pupil changed to a junior secondary 
school. Apparently when t 


hese children find that they are in the 
wrong school they simply leave at the minimum leaving age. 
Only 12 pupils (6 boys and 6 girls) out of the z,o12 who went 
to junior secondary schools changed to a senior secondary school. 
АП had spent a complete year in the first type, and all had Ho 
repeat the first year in the second type; so that for all these pupils 
there was a loss of a year. 


1 Circular No. 44, Scottish Education Department. 


1 
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The point is of sufficient interest to warrant the presentation of 
Miss Mitchell’s complete data: 


Length of Success or 


n Course in $ с 2 
- Quali- А к course in failure in 
Pupil fying Junior Senior senior senior 
No. score secondary secondary secondary | secondary 
(sigma) school school school school 
I TISSI Commercial French- Still on roll Success 
German 
2 +1528 Girl technical | French Still on roll Success 
3 +123 Commercial French 13 years Success 
4 “1-20 Commercial French Still on roll Success 
5 +119 Commercial French I year Success 
6 T1:2 Commercial French- I year Success 
Handwork 
7 +1-00 Commercial French- | Still on roll Success 
German 
+ -94 Commercial French Still on roll ^ Success 
9 + 79 Commercial French- Still on roll Success 
Handwork 
Io + -65 Boys technical French Still on roll Failure 
II = :05 Boys’ technical | French Still on roll Failure 
12 = :02  Boys' technical French I year Failure 


"The Qualifying score is that on our complete battery, and it is 
comforting to note that with our pass-mark of +-70 our prediction 
would have been correct for every one of these pupils. We should 
certainly have discouraged the last two from making the change. 
Nine out of the 12 have made good in the senior secondary school, 
and 6 are likely to complete the course. In the light of these 
figures it would seem that the flexibility of our system is not so 
Breat as is sometimes thought. 


CHANGES OF COURSE WITHIN SENIOR AND JUNIOR 
SECONDARY SCHOOLS 


Miss Rodgers's questionnaires were given 1 year and 8 months 
after the pupils had started the secondary courses, and certain of 
her questions bring out the flexibility of the system within the 
two types of school. 

She found that of the 391 pupils in her senior secondary group, 
58, or 14:8 per cent., had changed from one course to another, 
the main change being from a two-language to a one-language 
type. No less than 76 per cent. of the 58 changes were of this 
kind; 9 per cent. changed from one to two languages; and the 


4% 
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5 ical or 
remainder were mainly due to the dropping of technic 
domestic subjects. upils 

Another дегер point was that about ghe M заса 
were still dissatisfied with the course they were in. felt that they 
discontents were in the two-language courses br rude taken à 
would have been happier and more — t “i Indeed, 15 
technical or commercial course, usually the ded a note to the 
pupils, who were satisfied with their choice, x improve by 
effect that their literary courses would have Cen f outspoken 
the addition of some commercial work. ‘This adding ong лал 
critical notes was a common feature of the қов бейе e 
read the pupil’s educational views with an interest no 
with admiration for their soundness. as 66 the 

These results confirm the conclusions recorded later dett 
number of pupils who should be encouraged to 5. a opularity 
courses. They also suggest that the slow gri T of the 
of the newer types of secondary course is not үне those who 
pupils. Two further points that may be of interest to secondary 
plan the courses are that a large number of senior that coms 
pupils taking a commercial course expressed the view uw nan) 
mercial subjects should be started in the first year, and t senior 
pupils chose a junior secondary course rather Ex a coul 
secondary commercial course because, in the former, they 
begin the commercial subjects earlier. ly 17, 01 

Of the 420 pupils in the junior secondary group only sible 
4 per cent., had changed their course. Practically ale com- 
transfers were represented, the commonest being from t 2 Ве 
mercial to the technical. Seventeen pupils were still dissa 
with their course, :onnaires 

Many of the facts elicited by Miss Rodgers's questionn 
have a bearing upon the problem of the omnibus school. 


GE 
COMPARISON or THE Four GROUPS AT THE QUALIFYING STA 


QUALIFYING SCORES 


'The four Broups were senior secondary, junior secondary, back- 
ward and retained, and the distributions of the average scores OD 
our complete battery are shown in fig. 57. f 

These figures show the way in which the nation's resources 0. 
talent distribute themselves when almost free choice of school an 
course is left to parents and pupils. They are puzzling as well as 
illuminating. 
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The broken vertical is drawn at +-70 sigma, the point above 
Which the candidates for a senior secondary course have more 
than a бо per cent. chance of success. We may therefore assume 
that all pupils above this line have the ability to profit by a senior 
Secondary course. A glance at the histograms then shows that, 
while the 6 best pupils have all gone to the senior secondary school, 
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^ Fic. 57.—Distribution of Qualifying scores for the senior secondary, junior secondary, 
backward and retained groups. 


a very large proportion of those who had the necessary ability have 
not taken advantage of this educational opportunity. 

The next striking point is the tail of the senior secondary group 
and the distance it stretches below the success border-line, 
penetrating to near the bottom of the junior secondary group and 
to about the mean of the backward group. 

The lower part of the distribution of the backward group is in 
conformity with expectation, but those in the top intervals would 
seem to have deserved a better fate. Why, one wonders, should 
6 pupils with attainments up to the average of the whole Qualifying 
group be sent to these special classes? For only one did we 
find an explanation: the girl simply refused to go to a junior 
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secondary school. Of the other 5, one was promoted toa UN 
secondary course, the others were retained in the backward class P 
where the head teachers felt that their educational needs wer 
satisfactorily met. > 

Тһе терше of the retained and secondary groups 15 ало 
incomprehensible. "Three pupils who fall within the TS 4 Е 
cent. of the whole group are retained, while 5 who fall in 
bottom 16 per cent. are sent to senior secondary courses. Мб 

Тһе means and standard deviations of the Qualifying scor 

the four groups were as follows: 


Group N M 9 
Senior secondary . 452 1-102 sigma +67 sigma 
Junior secondary . 1,075 + 146 65 
Retained В ы 278 -1-167 65 
Backward . s 316 -1:429 58 

INTELLIGENCE 


the 
In an early preliminary study made before the gn wt 4 
retained group were available, Mr Robert Robertson fou 
following results for intelligence: 


Group Mean IQ 
Senior secondary 8 2 115:7 
Junior secondary Р a 102-0 
Backward . " у 84-0 


HEALTH, INDUSTRY AND INTERESTS 


Mr Robert Robertson found the percentages of the groups his 
were awarded the gradings A, B, C, D and E in each of t i 
factors. "То reduce the results to a simpler and clearer form W' 
calculated average scores by giving five points for an A, four for a B, 


and so on: 
Group Health Industry Interests tal 
Humanistic Realistic Practica 
Senior secondary 3:32 3:66 3:63 3:69 3:48 
Junior secondary 3:22 3:22 3:20 324 3:24 
Backward. c 3:13 2-30 2-15 246 247 ° 


These results have not the reliability which would warrant an 
attempt to calculate the statistical significance of the differences 


ы Se 
See eS „щш 
— 
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Shown. Yet the figures are not without interest, whether we 
compare the rows or the columns. In interpreting them itis well 
to keep in mind that the score for a normal distribution would 
be 3-00. 

In health the senior secondary pupils are the highest, and the 
Standard falls progressively from the junior secondary to the back- 
Ward groups, a result which bears out the view that intellectual 
and physical fitness are associated. This is probably the most 
reliable of the results, for it was based on careful comparable 
gradings by the School Medical Officer. For industry, where the 
fall from group to group is more striking, we have to realise that it 
must be difficult for the teacher to grade without being influenced 
by the pupil’s school attainment. Also, the disentangling of cause 
and effect would be possible only with a much more careful 
technique than we were able to adopt. 

We have already given reasons for distrust of the reliability of 
the interest gradings, but from the first row we see that the pre- 
dominant preference of the senior secondary group is not for the 
humanistic but for the realistic. This may bring some encourage- 
ment to those who believe that the curricula of these schools are 
too literary and linguistic, and that realistic and practical subjects 
should have a more prominent place in some of them. These 
Senior secondary pupils are, indeed, more interested in practical 
subjects than the pupils in the junior secondary group, whose 
tastes are more evenly distributed. On the whole, however, the 
interests of the latter are rather realistic and practical than 
humanistic, 

The interests of the backward group presumably lie elsewhere 
than in the world of the school, for they are well below 3:00 in all 
types of subject. Yet they make their highest score in the practical 
group, which supports the belief that the curriculum for pupils of 
this type should consist largely of manual activities. 


Comparison ОЕ THE SECTIONS TAKING DIFFERENT COURSES 
IN THE SENIOR SECONDARY SCHOOLS 


The main interest of this comparison is that it allows us to judge 
how far the newer senior secondary courses had, by 1935, begun 
to attract the abler pupils. When the Latinless section of the 
French /усёе was abolished in 1923, a Professeur protested that this 
section had formed a convenient depository into which, with a 
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7 D “ys do 
light heart, were dumped the pupils with insufficient ability to 
wall in на апа + asked КЫЛЫП. * What is now to be Е. 
with these undesirables? To find whether the. pecus M 
technical courses perform the same service in Scottish schoo 
prepared the following comparison: 


4 2 Mean Qualifying Mean IQ 
Section N уте 

Latin-French . т X +1-367 sigma Бес: 

French-German . IIO TI:315 be 

French-Technical . 74 4- 1:006 01 

French-Literary . тоо + +619 5 


с 
In the French-Technical category we included French Бош M 
and French-Commercial, and it is clear that such ET The 
beginning to attract a fair proportion of the clever pup! French: 
dumping-ground for weak entrants would appear to be the 
Literary course. 


ЕЛ 
GIRLS 
Comparison or SECTIONS TAKING Boys’ TECHNICAL, 


p JUNIOR 
TECHNICAL AND COMMERCIAL Courses ім THE J 
Srconpary Scuoors 


on saat dar, 
The mean IQs and Qualifying scores for the junior secondary 
courses were as follows: 


Qualifying score IQ 
Course М ы M 5 
Commercial +-681 sigma “59 sigma 110:2 10:3 
Boys? technical +078 -60 IOI:2 103 
Girls’ technical --ой “бо 98-7 95 


"Тһе commercial cou 
the best of the junior s 


were 114-4 and 106-1. 
secondary schools is theref: 
French-Literary group i 
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e COMMERCIAL 
— —— BOYS' TECHNICAL 
—GIRLS' TECHNICAL 


FREQUENCY 


QUALIFYING SCORE 
(sigma) 
Fic. 58.—Distribution of Qualifying scores for courses in junior secondary schools. 


System or RETENTION 


The retention system is a departure from the ‘clean cut,’ which 
provides for the pupil who, if promoted, would have to go to a 
backward class, but who would be fitted for a normal secondary 
course if his attainments were increased by a further period in 
the Qualifying class. Our data provide some information as to its 
practical operation. 

Below we give details of the duration of the period of retention 
before promotion for the retained pupils in our group: 


Number who were fit for 


2 5 Fit onl 
D Senior Junior Day School y 
Retained for Leaving Leaving Certificate эша b 82 а Total 
Certificate Certificate (Lower) 80090 bacpvar 
т year 6 months . .. T 3 3 - 3 
I year à 2 1 6 23 54 43 97 
6months . ` 1 3 16 8o 77 157 
No post-primary 
career . 97. fid 48 z. „= 21 21 
Total . 2 9 42 137 чї 278 


E 


208 SELECTION FOR SECONDARY EDUCATION 
The following were the courses taken by pupils after retention: 


Number who were fit for 


Fit only 
Senior Junior Day School Shorter for Total 

Course taken Leaving Leaving Certificate course backward 
Certificate Certificate (Lower) class е 
Senior secondary . H 3 6 6 ЗЕ 109 
Junior secondary . 1 6 36 108 35 142 

Backward . s ss 23 
No post-primary zi 21 
career > кз va 22 z 2” 
Total . E 9 42 137 141 


The second set of data shows that the system serves die 
purpose. If we assume that all the retained group yer Mer 
gone to backward classes in a clean-cut system, we see ip Кес 
saved about half of them from that fate. While many о ER 
would no doubt have been transferred later from the backw: 


group, the advantage of keeping them in a normal Qualifying gtUP 
is considerable. 


IMPROVEMENT AFTER SIX MONTHS’ RETENTION 


We present fig. 59 partly to give an idea of the improvement a 
standing resulting from six months’ retention, and partly as m 
example of a case where we could not see why retention ен 
thought necessary. This girl, who was first presented at 11 yea 2, 
falls above our pass-mark for admission to a senior secondary n 
She is described as persevering and self-reliant, and the report © 
the home is ‘considerable parental interest.’ 5 е 

After six months’ retention she improved her Qualifying e 
froni +-82 sigma to 41-20 sigma, and then proceeded to t? 


c 27512 е 
commercial course at a Junior secondary school. "Thereafter, sh 
transferred to a seni. 


е 
От secondary school where she had to take th 
first-year course, thus losing one and a half years in all. 


RECOMMENDATIONS WITH REGARD To RETENTION 
A detailed study of the data Suggests the following tentative 
conclusions regarding the system of retention: 
The system of retaining pupils for six months is valuable in 
certain cases, 
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----- АТ FIRST PRESENTATION 
fesso eee AFTER SIX MONTHS RETENTION 


JUN. SEC. COMMERCIAL 


SEN. SEC 


L__!& Se E Tes Qa A Tas Av Tp Ind.H Fc. I 


Fic. 59.— Case where retention was probably unnecessary. 


Another striking point is that the number who, after retention, 
аге found to be fit only for backward classes is rather high. Ыб 
these pupils could have been detected at the first presentation 
they could have been sent to the backward classes at once, thus 
relieving the Qualifying teachers of some dead-weight, and giving 
à sounder educational treatment to the pupils themselves. It is 
Certainly not to their educational advantage to keep them at a 
hurdle which is too high for them, and run the risk of further 
sapping their self-confidence. This is an aspect of the selection 
problem which would repay further attention by Education 
Committees. Like many others, it raises the question of the 
desirability of a preliminary sifting of the material at an earlier 
stage in the primary course and of a development of special classes 
within the primary school. 

If, in consultation with their head teachers, Committees will fix 
their policy in regard to backward classes, the type of course to be 
offered, the type of pupil who should be sent to them, and so on, it 
would be possible to fix a border-line as for senior secondary 
courses. At the moment the attitudes of the head teachers to 
these classes vary so much that the problem is insoluble. 
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Tue BAckwanp Group 


The distribution of Qualifying scores for this group, shown in 
fig. 57, would lead us to suspect that a number of pupils had been 
relegated to the category in error. This is confirmed by 
` judgments of the teachers who taught them in the backwar 
classes: 


Judgment of teachers as to degree of fitness N 
of pupils sent to backward classes 
Fit only for backward class . ” $ . 2 283 
Fit for junior secondary course after retention . . 26 
Fit for junior secondary course at time of trans- 
ference to backward class . , E . . 7 
obl s- 42 x i BIO 


The number of errors in placement, while considerable, was 
therefore not unduly large. 


CHAPTER XXI 


PERCENTAGE ОҒ PUPILS FITTED FOR (a) А 
SENIOR SECONDARY EDUCATION, (0) A JUNIOR 
SECONDARY EDUCATION OF JUNIOR LEAVING 
CERTIFICATE LEVEL 


Tuts old problem, whose obituary notices have so often appeared 
in the writings of educationists, has an administrative vitality 
which has preserved it amidst the constantly changing ideas of 
Secondary education. But present nomenclature makes it difficult 
Sven to state precisely what the problem is; and any answer to it 
is destined soon to go out of date. All we can do is to give estimates 
for secondary courses of the types that are offered now and with the 
standards that now obtain. 


SENIOR SECONDARY EDUCATION 


DISTINCTION BETWEEN POTENTIAL AND AcTUAL SUCCESSES 


From our data we can estimate the percentage of the Qualifying 
group who would actually have been successfulin a senior secondary 
course. The difference between this and the percentage who 
actually took and succeeded in such a course is a measure of what 
may be regarded as waste of national talent. T 

In assessing the waste, from the point of view of the individual 
Pupil, we must bear in mind that the difference is mainly in length 
of course, and that the needs of many clever pupils may be best 
met in the junior secondary school. But from the statistical stand- 
point the measure is too lenient. 


A TYPICAL CASE 


In fig. 60 we give the profile of a boy with a Qualifying score of 
+180 sigma who went to a junior secondary course and was 
judged by his head master as one who would have been a certain 
failure if he had gone'to a senior secondary school. 

211 
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i i ilure 

The head master's view that this boy would have been a me 

in a senior secondary course was based upon knowledge of his u Xx 
favourable home conditions, where there was a constant strugs 


IQ Ge E Tes ба A Tas Av. Tp 


Fic. 60.—A pupil who had the ability and attainment for success in a senior 
secondary course but who would have been a failure. 


; З > in his 
with poverty and where he received no encouragement pue 
studies. In interpreting the profile it has to be remem?” . 


А is junior 
that the follow-up marks are relative to the mean of his ju 
secondary course. 


ANALYSIS OF 78 CASES WITH QUALIFYING SCORES OF 
"CF I*I SIGMA AND OVER 


In discussing individual pupils with head masters of the junio? 
secondary schools we frequently came upon instances of the above 
type. Тһе child had the necessary ability and attainment but 
would have succumbed to the resultant drag of his environmental 
conditions. Sometimes the source of the trouble lay in the 
poverty of the home, and the parental eagerness for the child to 
make his contribution to its meagre resources, In such conditions 
the child had no home encouragement in his further education; 
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often a positive antagonism, the offspring of sheer necessity. 
Frequently we were told that the child was a member of a large 
family living in a house affording no facilities for the home study 
demanded of the senior secondary pupil Even when the home 
conditions were moderate or good, other strong forces of the 
environment seemed to be arrayed against the pupil; his friends 
were all leaving school at 14 to become van boys or message boys, 
with their evenings free and a little money in their pockets, and 
he wished to leave with them. The attraction of the senior 
secondary school was far too weak to overcome these opposing 
pulls, 

In the junior secondary courses there were 78 pupils with 
Qualifying scores of +1-1 sigma or over who would have been 
failures in the senior secondary school if they had gone there. Six 
out of 7 pupils in the senior secondary group with this Qualifying 
score are successes; so we may take it that these 78 pupils had 
the necessary ability. Their cases were made the subject of 
independent study, and the following reasons why they would not 
have been successful in the senior secondary school were found: 


Reason for failure ң М 
Unfavourable home conditions . қ в 18 
Lack of interest : desire to take up work . 16 
Personal qualities : laziness, delinquency, etc. 8 
Ill-health. Э T a x M s 4 
No suitable course in senior secondary school 3 


No known reason, apart from the fact that the 
5 каре йя! 
pupil's performance in the junior secondary 
school was unsatisfactory . . . 29 


Total. s s o 78 


The categories overlap; and when more than one cause was 
operative we placed the pupil in accordance with what the head 
teacher thought to be the main one. 

It is difficult to disentangle the complicated motives that weigh 
with these young adolescents. In 16 cases the pupil was not 
interested in school-work and wanted to take up employment. 
Which is the cause and which the effect? Is there a natural 
developmental tendency which impels certain pupils at this age 
to leave the artificial sheltered environment of the school and seek 
the real life of the workshop? Or is it merely that the pull of the 
environment, or the lack of reality and attraction in the present 
courses, creates such a tendency? "The whole problem is one of 
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Ат В TETUER igation 
grave social importance and merits a more scientific кы к 
than we were able to make as a side line to an already ех 
Inquiry. 


True Measure or WasTE 


If we wish a complete estimate of the waste we must -—À p. 
percentage who actually succeeded in a senior secondary ел 45 they 
from the percentage who would actually have been success em pi^ 
had gone there, but from the percentage who have the ability d 
attainment necessary for success. Even thus we do not get a 
full measure of the waste. In the first place, we get no rp. 
tion. as to the number of pupils who failed to reach the leve o 
attainment for success in the senior secondary course Бейше Е 
adverse conditions during their primary course. In the A et 
place, we have no way of measuring the increase in the pecu. 
potential successes which would result if the senior sec uu 
courses had the degree of reality and attractiveness to 
many modern educationists aspire. 


THE THREE ESTIMATES REQUIRED 


А 1 Op қ R hree 
It will be conducive to clarity if, at this stage, we list the t 
estimates at which we shall aim. These are: 


(1) Percentage of the Qualifying group who had the € 
and attainment necessary for success in а 586 
secondary course. lina 

(2) Percentage who would actually have been successfu 
senior secondary course. i 

- (3) Percentage Ав actually were successful in а s 
secondary course. 


(1) Percentacr Or Purtzs wHo nap тнк Авплтү AND ATTAIN- 


MENT NECESSARY FOR Success Іх А SENIOR SECONDARY 
CoursE 


The second and third of the above percentages are based upon 
judgments by the secondary head master as to whether the pup. 
had more or less than a 50 per cent. chance of being successful. 
We have to base the first upon a statistically determined 50 рег 


cent, chance of success; and since this is a different thing we shall 
first try to make the distinction clear, 


1 
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, 
MEANINGS OF “о PER CENT. CHANCE OF SUCCESS 


When we asked a head master to decide whether a pupil had more 
than a 50 per cent. chance of success his decision was an absolute 
one, in the sense that it took account only of two things: the pupil, 
with his individual abilities, attainments, interests and environ- 
mental conditions, and the standard of the Leaving Certificate 
examination. It had no reference to any particular entrance test 
or to the probable fate of other pupils with the same entrance score. 

hen we stated that a senior secondary pupil with a score of 
+°70 sigma had just a little more than a 50 per cent. chance of 
Success, we used the phrase in a different sense. The judgment in 
this case was a statistical one, based upon the fact that half of the 
Pupils in that group with an.entrance score of + “695 sigma on that 
attery are successes and half are failures. It had reference to a 
Particular entrance test and to other pupils with the same score. 
If the battery happened to be a poor one, the point at which the 
Pupil would have an even chance would be different. 


LEVEL OF ABILITY AND ATTAINMENT NECESSARY FOR SUCCESS IN 
SENIOR SECONDARY COURSE GIVEN FAVOURABLE ENVIRONMENTAL 
AND OTHER CONDITIONS 


As usual we use as Qualifying scores the average on the complete 
battery ТО+О+8-+ Ts. Any point on this scale, such as +:695 
Sigma, represents a definite objective standard of ability and 
attainment. Now we have found that pupils in the senior 
Secondary group who are at this level on this battery have just 
ап even chance of succes. We therefore assume that, if the 
extraneous conditions which affect success were as favourable for 
the whole Qualifying group as they are for the senior secondary 
group, then +-695 sigma would be the point at which half of the 
pupils in the whole group would be successes and half failures. 

е also assume,that, in these circumstances, if the whole group 
had gone on to the senior secondary school the correlation between 
Qualifying scores and success would have been `8. 


THE BEST ESTIMATE OF PERCENTAGE ОЕ SUCCESSES GIVEN 
CERTAIN CONDITIONS 


If, for a certain battery, the normal correlation between Qualify- 
ing score and success is ---8, and if a pupil with a score of +-695 
sigma has just an even chance of success, what is the best estimate 
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we can make of the percentage of pupils who would be ae 
in the secondary course? At first sight it would seem that a |. 
have to do is to find the percentage of pupils who, in a ER 
distribution, lie above +-695 sigma. This, as a maero ^ ү 
is 2474 per cent. We are therefore tempted to reason t е; ч 
per cent. of pupils have more than an even chance of segs p 
that this could legitimately be taken as the percentage w. ЖЕРІ. 
the ability and attainment which fits them for senior seco 
education. - 
We must, however, bear in mind that we are here using А 
phrase *even chance of success? in the second, ог statistical e 
At +-695 sigma the pupil has an even chance as predicen Ae 
battery which gives a correlation of +-8. With a ew a о, 
the correct pass-mark would be different, and we should g 
different ercentage. | 
We * that a better estimate may be found as follows: 


QUALIFYING SCORE 


SUCCESS IN SECONDARY SCHOOL 
Fic. 61.—Best estimate of Percentage of successes. 


AB is the correct pass-mark of +695 sigma. The volume 
ARB is the percentage of pupils who, as predicted by the battery, 
have more than an even chance of Success. To find the best 
estimate of the required percentage of successes we have to find the 
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line CD which, with this correlation, would separate successes 
from failures. This is easily done,! and we find that it lies ata 
distance of + :556 sigma above the mean. Іп a normal distribu- 
tion the volume CBD would therefore be 28-9 рег cent.; and 
28-9 per cent. of our group of 3,021 is 873. А 

Actually our distribution of Qualifying scores is not quite 
normal. "The percentage of pupils with scores above +:695 
sigma is not 24-4 but 2577. If we take this into account we find 
that the volume of CBD would be 30:1 per cent., and the number 
of Successes in our own group, 909. 

It may be of some general interest to mention that of the 776 

Pupils who fell above +695 sigma on our Qualifying scores, 339 
Went to senior secondary schools, 434 to junior secondary schools, 
2 were retained, and 1 actually found his way into a special class 
for backward children. " 
. Vr estimate, of whose statistical weakness we are fully conscious, 
15 that the percentage of the Qualifying group who had the ability 
and attainment necessary for success in a senior secondary course 
15 3041. 


(2) Percentace wHo WOULD ACTUALLY HAVE BEEN SUCCESSFUL 
IN A SENIOR SECONDARY COURSE 


We have already explained how we found the numbers of pupils 
who, in the opinion of the secondary head teachers, would have 
ееп successful in courses leading to the Senior Leaving Certificate, 
Junior Leaving Certificate, etc., and the complete data are given 
in Table XLIV.2 Y 

Into this Table, one of the most important in the Report, is 
condensed a large amount of information. All we require at the 
moment is the second column, from which we see that the number 
of those who would actually have been successful in a senior 
Secondary course is 464. Of the 452 who embarked hopefully on 
such courses, 320 were graded as successes and 132 as failures, while 
142 successes came from the junior secondary group. 

Our estimate is that the percentage of the Qualifying group who 
would actually have been successful in a senior secondary course 
їз I4. 

The percentage is disappointingly small, but from our experience 

! Equation of AB is y= +-695. Equation of the regression line of the x-arrays 


is y=1-25x, These lines intersect at (+556, ++695). 
$ P. 218, 
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of the careful and competent way in which the head teachers made 
their final gradings we are convinced that it has a high degree of 
reliability. 

TABLE XLIV 


Numbers of pupils who would have been successful in various 
levels of secondary education 
y 


Number who would have been successful in 


Fit only 

Group N Senior Junior | Day School [s for special 

Leaving Leaving | Certificate Кр classes for 

Certificate | Certificate | (Lower) СОЁ backwards 
M| o (3) С) ө | © 
Senior secondary 452 320 341 430 448 * 
Junior secondary | 1,975 142 615 1,247 1,933 A 

Retained . 2 278 2 10 43 138 abs 

Backward.- .| 316 - “& 33 283 
Total  .| 3,021 464 966 1,720 2,552 469 


(3) PERCENTAGE WHO ACTUALLY WERE SUCCESSFUL IN A 
бемтов Srcowpany Course 


From column (2) of Table XLIV we find that of the whole 
group of 3,021 only 320 actually took and were successful in 
senior secondary course. ho 

Our answer is that the percentage of the Qualifying group W 
actually were successful in a senior secondary course 15 10:5. 


Percentace or Toran POPULATION WHO wOULD ACTUALLY 
BE SUCCESSFUL IN a SENIOR SECONDARY COURSE 

Тһе above results are given in the form in which they will X 
most useful to Scottish Education Committees. But considerable 
general interest attaches to the percentage of the total population 
who would be successful in a senior secondary course. This is по 
quite the same thing as the percentage found in (2) above; for 
some of the Qualifying pupils are presented more than once. 
То get the percentage of the total population we require E 
group that is ‘pure,’ in the sense that every pupil comes into it 
once and not more than once. Sucha group is that promote 
to the various post-primary courses in а particular year; and, 
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after making slight adjustments in respect of a small proportion 
(21 out of 3,021) who had no post-primary career, we find the 
result to be as follows: 

As conditions are, 16-7 per cent. of the total population would be 
Successful in a senior secondary course. 

In Chapter II we found some evidence indicating that the 
children in the City in which the Inquiry was made were lower in 
intelligence and scholastic attainment than pupils in certain Areas 
in England. Few Scotsmen will hesitate to deduce from this that, 
for Scotland as a whole, the above percentages might have to be 
Somewhat increased. 


Соглкстер RESULTS ror SENIOR SECONDARY EDUCATION 
We can now collect into a single Table the complicated answer 
to the deceptively simple question in the title of the chapter. 


TABLE XLV 
Fitness for senior secondary education 


Group N Per cent. 

Total Qualifying group з ; „| 292 тоо 
Pupils who had the ability and attainment 

necessary for success in a senior secondary 

Course Е $ Я А х қ 909 301 
Pupils who would actually have been success- 

ful in a senior secondary course E з 464 154 
Pupils who actually were successful in a senior 

secondary course . : = r 5 320 10-6 
Pupils who took a senior secondary course and 

were failures Е » p . Б 132 44 


SECONDARY EDUCATION FOR ALL 


Table XLV does not solve this problem, but it gives some facts 
Which clarify it. If by secondary education we mean the courses 
at present offered in our senior secondary schools, with their 
present content and standard of work demanded from the pupil, 
then we must conclude that secondary education should not be 
for all but only for, at most, 30-1 per cent. of the Qualifying pupils. 
Even if there should be a considerable extension in the provision 
ОЁ newer courses, with Art, Music, etc., as the main subjects, we 
doubt if the percentage could be much increased. 
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EpucarioNAL WASTE 


ion 
Yet while we cannot support the doctrine of secondary ee 
for all interpreting it in the narrow sense белила HA Pa 
Table XLV shows that the benefits of such education bi. 
much more widely spread, and that their em ы: icon 
results in an educational waste which should be eee der. 
delay. We fully realise that the 4-4 per cent. of pupi sud лобе 
a senior secondary course but turned out to be failures m e 
written off as an educational bad debt; also that рер тош 
ability, who go to a junior secondary course, have a "s ranks in 
technical and commercial colleges to the commissione Бю 
industry and commerce. Ас the same time there is mu nation 
said for the view that, in their own and in the wider nd more 
interest, they should have had the benefits of the richer а 
liberal culture of the senior secondary school. : s follows: 
With these reservations we may state our conclusions a 


А da 
At present the number of pupils fit for a senior ped p 
course, who embark on such a course, is 10:6 рег 
the Qualifying group. istin 
By proper w of ae — schemes and of the best able x 
methods of selection and guidance we should be 
raise this to 15-4 per cent. ә 
То reach the final goal of 30-1 per cent. we require: T 
(а) A far-reaching improvement in social and env 
mental conditions. қ of the 
(P) An increase in the variety and attractiveness 
senior secondary courses. 


PERCENTAGE оғ Quatiryinc Рориз FOR ууном Provision 
SHOULD BE MADE IN SENIOR Srconpary SCHOOLS 
; PR ion 
For what percentage of Qualifying pupils should an pue 
Committee make provision in its senior secondary schools? 
is а difficult question, the answer to which depends upon: 


© Which of the figures in Table XLV (ooo, 464 or 320) We 


accept as the number of pupils fitted for a senior secondary 
education. 


The group from which we make the selection. 
The battery used in selection, 
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In the City in which the Inquiry was made there were 320 pupils 
/Who took a senior secondary education and were successful in it. 
As we have seen, there is no possible way of ensuring that these 
320 and no others will be admitted; and the number we admit 
Will depend on whether we make the selection from a group of 
candidates, all of whom are desirous of a senior secondary educa- 
tion, or from the complete Qualifying group. The former being the 
more probable alternative, we shall base our calculations upon it, 
and give results which will probably hold for any district in 
Scotland, қ 

If the number of pupils who desire, and would be successful in, a 

senior secondary course is 320, the number of places to be provided 
is 335. Of these 335 pupils, 34 would prove to be failures; and 
in addition we should reject 19 pupils who would have been 
successful, 
,, Putting the above result in percentage form, we may say that 
if 10:6 per cent. of the Qualifying pupils desire a secondary educa- 
tion and are fitted to profit by it, places should be provided in the 
senior secondary schools for 11-1 рег cent. of the Qualifying 
Broup. 

If by improvement of social and educational conditions it should 
ever be possible to raise the percentage to 30-1, secondary places 
would be required for 31:2 per cent. of the Qualifying group. 


JUNIOR SECONDARY EDUCATION UP TO JUNIOR 
LEAVING CERTIFICATE LEVEL 


Of the three estimates mentioned on p. 214 the first two were 
obtained by the methods already described. In the percentage 
who actually were successful we included, in this calculation, only 
those who obtained the Junior Leaving Certificate. It was not 
possible to use the corresponding figure with the senior secondary 
group. If we had done so, the estimate of educational waste 
would have been even more alarming; for many of the 10:6 рег 
cent. of potential successes would not complete the course. 


PERCENTAGE WHO ACTUALLY OBTAINED THE JUNIOR LEAVING 
CERTIFICATE 


In the senior secondary schools pupils who are proceeding to 
the Senior Leaving Certificate do not sit for the Junior, so we give 
statistics only for the junior secondary group: 
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Course 5 
se: 
Boys? and дігін Commercial ww 
technical E 
Рае . 2 . 58 133 t$ 
Failed 4 я > `9 20 
Not presented because 46 
notuptostandard . 16 10 
Did not sit for other 18 
reasons than above . 18 e n 
Left before examination 1,491 220 , 
Total 1,592 383 1,975 
Percentage of Certifi- 97 
cates obtained & 3:6 347 


he 

То secure our comparative figures we have now to ape 
191 junior secondary pupils who obtained the pa ae may 
Certificate those pupils in the senior secondary schools calculate 
legitimately be regarded as having obtained it. We “| who left 
this in a reliable way. First of all we must exclude a from the 
too early; for the remainder we have definite judgments 1T mina- 
head teachers as to whether they would have passed the € oe 
tion if they had sat it. Reckoned in this way, the AER re take 
senior secondary passes to be added is 269. We тау ther т ndary 
it that the number who successfully undertook a junior seco 
course was 460, or 15:2 per cent. of the Qualifying group. 


ION 
CorrrcrEp RESULTS FOR Junior Seconpary Epucat 


TABLE XLVI 
Fitness for junior secondary education up to level of 
Junior Leaving Certificate 


Group 


Total Qualifying group . E 5 . 

Pupils who d the ability and attainment 
necessary for success in the Junior Leavin, 
Certificate 4 : i : à $ ^ : 

Pupils who would actually have been successful in 
the Junior Leaving Certificate 3 қ 

Pupils who actually were Successful in the Junior 
Leaving Certificate 

Pupils who took a Junior L 
course and were failures 


eaving Certificate 
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Comparison WITH RESULTS FOR SENIOR SECONDARY 
EDUCATION 


When we compare Table XLVI with Table XLV perhaps the 
most striking point is that the ability level necessary for the Junior 
Leaving Certificate is nearly as high as that for the Senior. 

The next point is that nearly all the pupils with the necessary 
ability would have gained the Certificate if they had completed 
the course. This was far from being so in the Senior Leaving 
Certificate courses, partly, no doubt, because their length might 
discourage a pupil willing to tolerate school up'to the age of 
15 but no longer, and partly, it may be, because the junior 
Secondary courses are more attractive to a large proportion of 
pupils. 

It must, of course, be kept in mind that the 15:2 per cent. 
regarded as having successfully undertaken a Junior Leaving 
Certificate course includes the 10:6 per cent. who are successfully 
taking the Senior Leaving Certificate course, For this reason the 
educational waste in the junior secondary schools is best assessed 
from the statistics given for the junior secondary group on p. 222, 
taken in conjunction with the fact that 61 5! of this group would 
have been successful in the Junior Leaving Certificate if they had 
completed the course. ОҒ these 615 only 191 actually did obtain 
the Certificate, showing that we have considerable leeway to make 
up in our preparation of non-commissioned officers for industry 
and commerce. 


DAY SCHOOL CERTIFICATE (LOWER) 


Since the Inquiry was started in 1935 important changes have 

een made in the organisation and nomenclature of post-primary 
courses and Certificates. In the main these do not affect our 
results, and we have, up to this point, used the new terms. For 
example, the Junior Leaving Certificate was previously called the 
Day School Certificate (Higher), but when the name was altered 
the Scottish Education Department intimated that the standard 
required would remain unchanged.? 


1 See Table XLIV, p. 218. 
* Circular No. 120, gth June 1939. 
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Тһе Day School (Lower) was previously awarded to pupils "d 
had satisfactorily completed a two-year course; but the 1939 yc 
provides that pupils who leave school without having satisfied the 
conditions for the award of a Junior or Senior Leaving Certificate 
shall be entitled to receive from the Education Authority a MA 
ment showing the particular stage of advancement reached at t 
date of leaving school. 

While our seti are thus out of date so far as the Day gaa 
Certificate (Lower) is concerned, we give a summary of ах 
which may have some historical interest, and will complete t 
picture of the conditions prior to 1939: 


Course 
rses 
Boys’ and Girl? Commercial ы 
technical 
Passed . T қ s 584 268 ES 
Failed . à i Р 31 je 
Not presented because not 8 
up to standard . 5 323 15 A 
Left before examination 628 61 9 
Did not sit for other than 6 
above reasons . қ 26 39 5 
"Total Р Я 1,592 383 1,975 
Percentage of Certifi- 
cates awarded К 267 700 431 


Some pupils who would have obtained the Certificate уйше п 
presented because they were proceeding to the Day Sc oh 5 
Certificate (Higher). ‘These form the greater part of those У 
*did not sit for other than above reasons; and even if we give 
the whole 65 the benefit of the doubt the picture is somewhat 
depressing. 

According to Table XLIV, 1,247 junior secondary pupils would 
have obtained the Certificate if they had completed the course; at 
most only about 917 out of 1,975 could be regarded as having 
reached this modest standard. 


Our border-line for success in the Day School Certificate 
- (Lower) was —:145 sigma. 
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PERCENTAGES FIT FOR OTHER Forms or Post-PRIMARY 
EDUCATION 


These comprise courses of a lower level than the Day School 
Certificate (Lower), and also special classes for backward pupils. 
Аз far as the latter are concerned we have little confidence in our 
results, for the head teachers had very varied attitudes to the 
backward classes. With this reservation we may say that about 
84:5 per cent. of the Qualifying group appear to be fitted for 
pormal classes and 15-5 per cent. for classes for backward pupils. 
The border-line separating the two groups was at — 1-085 sigma. 


= тай urse ? 
would you put him into a two-language or a one-language eo 


e 


CHAPTER XXII 


О 
PROPORTION OF SENIOR SECONDARY PUPILS 54 
SHOULD ТАКЕ А TWO-LANGUAGE COURS 


Wuar proportion of senior secondary pupils should take а 
language course? This is a question the answer to W Дал] 
relevant only to the present conditions in certain Areas in 5со Em. 
A French-Technical course could be made as stiff as a ee у. 
one, but this does not appear to be the general rule in this [P vidt 
А common arrangement is that pupils taking only one "er 
give extra time to some other subject already in the pork leds 
thuslightening the pressure. Frequently a change to this ud 
course is made only after a pupil has spent a year at the $ с 
language; and it was to devise a procedure to avoid this wa 
effort that we studied the problem. 


E URSE 
Decisions As To Fitness ror a Two-LANGUAGE Co 


h 

We dealt only with the senior secondary group, and i б 
individual pupil we asked the head teacher, at the end e what 
years, to give a decision on the following issue. If, knowing in 
you now know of this pupil, you had to enrol him in the first yea", 


It was a difficult question, and we should not have been Mp 
if the head teachers had hesitated to attempt an answer; yet We 
‘we considered the individual pupils the difficulty was less than 5 
had expected. There was little difference in the standards us i 
by the different head teachers, for the border-line was approx 
mately the same in each of the five schools. 


NUMBER JUDGED FIT FOR A TWO-LANGUAGE COURSE 


'The number of senior secondary pupils who entered проп, 


and the number who were fit for, а two-language course are 
as follows: 
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Group N Per cent. 
Total number enrolled in senior secondary course 452 100 
Number who took a two-language course . . 278 61-5 
Number who gave up thesecondlanguage. — . 35 y 
Number fit for a two-language course . 2-16 365 


Of the 278 who embarked on the two-language course only 165 
were іп the end judged to be fit. Even after the withdrawals the 
number taking the heavier course is still far too large. 


SELECTION For Two-Lancuace COURSES 


The wastage of effort on the part of teachers and pupils suggested 
by these figures is so considerable that Education Committees and 
head masters may desire to take some steps to reduce it. 

The best way to do this consists in finding the pass-mark for 
3—70" secondary pupils on the Qualifying scores for the complete 
Qualifying group at which the pupil has just an even chance of 
Success in a two-language course. "This was done by the methods 
already explained and comes out to be + 1:435 sigma, a standard 
which could be used by the secondary head master as a rough 
guide in his first classification of entrants. It should not, of 
course, be treated as a hard-and-fast line. Many pupils above this 
point, through lack of desire or of literary abilities and tastes, 
might be well advised to take only a single language. On the other 
hand certain pupils who are below this line might be admitted 
if they had special linguistic ability; but such admissions should 
be made only with some hesitation. In Chapter XXV we show 
how this border-line could be used in practice. 

Rough guiding figures are that about 7°56 per cent. of the 
complete Qualifying group, and 36:5 per cent. of the senior 
Secondary applicants, have the ability and attainment necessary 
for success in a two-language course. 3 


CHAPTER XXIII 


LEVEL OF INTELLIGENCE NECESSARY FOR SUCCESS 
IN THE VARIOUS LEAVING CERTIFICATES 


SENIOR LEAVING CERTIFICATE 


Тинвве problems are mainly of theoretical interest, for we trust that 
no one who has studied fig. 42! would use IQ alone as a basis О 
selection. We have also referred to the statistical pitfalls in try1n8 


"INQUIRY 
OTIS ADVANCED 


PERCENTAGE OF FAILURES 


INTELLIGENCE. QUOTIENT 


« 62.—Curves showing the percentage of failures in the Senior Leaving 
n Certificate for different IQs. 


Fic. 


to deduce, from such data as we give below, the percentage of pupils 
fit for secondary education. Certain results have, however, bee? 


published on the subject,? and we give our own for comparison- 
1 P. 164. 
2 For example, F. M. Earle, Tests of Abili Courses 
Publications of the Scottish Coun of Ability for Secondary School 


cil for Research i і ndon: 
University of London Press, Ltd., 1936. Bi in Education No. X. Lo 
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The graphs in fig. 62 show how uncertain would be any pre- 
diction of success based оп IO alone. Pupils with very low IQs 
may pass; pupils with very high IQs may fail. The following, 
indeed, is the only answer which we can give to the question on 
the basis of the Inquiry results: 


Pupils with an IQ of 108 have an even chance of success, 
Pupils with IQs under 91 are almost certain to fail; those 
With an IQ over 130 are almost certain to pass. 


Two points must be kept in mind in interpreting these results: 


Our test of success was that the pupil would get a minimum 
Leaving Certificate in a suitable course in six years, 

Our IQs are for tests with a mean of roo and a standard 
deviation of 15. 


Comparison wirn RESULTS OBTAINED BY USE OF 
Oris Apvancep Test 


For a number of years Miss Young collected results obtained by 
students in the application of the Otis Advanced Test to Leaving 
Certificate classes where the examination results were known. Her 
Study of these, in which she had the assistance of Mr Hugh 
Stewart, B.Sc., and Mr David С. Johnstone, M.A., shows that 
a satisfactory solution of our problem cannot be found by this 
method unless the numbers are much larger than the 595 already 
obtained. The main difficulty, which is increased by the practice 
of not presenting all the pupils in the Leaving Certificate classes, 
is that the numbers with the lower IQs are too small, and that 
the group is too highly selected. The numbers who passed and 
the numbers who failed in the Senior Leaving Certificate for the 
various IQs are given on P. 230. 


95 has passed. These actual results of presentation at the examina- 


the success gradings were prophetic, 

The data also bring out the statistical difficulties in determining 
from them the point at which the child has an even chance of 
Success. A fair guess may, however, be made by smoothing the 
curve of percentage of failures at the parts where it is reliable and 
completing it in accordance with what seems to be its general 
tendency. This is the broken line in fig. 62. 
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IQ from Otis Number who Number who 
Advanced Test passed failed 
135 I 
130-134 12 
127-129 31 
126 18 2 
125 “29 1 
124 39 4 
121-123 107 1 II 
118-120 88 27 
115-117 68 26 
112-114 38 26 
109-111 19 16 
106-108 13 + 
103-105 4 4 
100-102 5% 2 
97-99 1 3 
96 «s 
95 1 
Total 469 126 


The results are as follows: 


Pupils with an IQ of тод have an even chance of е nnd 
Pupils with IQs under 92 are almost certain to fail: t 
with IOs over 128 are almost certain to pass. 


` The conformity with our Inquiry results is therefore reasonably 
“close. 


2 


JUNIOR LEAVING CERTIFICATE 


"Тһе problem is of even less practical value for the Junior Less 
Certificate, and it did not warrant the labour of making specia 
distributions. We know, however, that 615 out of the 1,975 
junior secondary pupils were successes, and we know the correla- 
tion between IQ and success for this group. We сап therefore 
obtain an estimate of the border-line by the method described 
in Chapter XIX. It comes out to be an IQ of 109. Р 

"The fact that this is higher than that just found for the Senior 
Certificate in the senior secondary group will occasion no surprise 
to the reader who has grasped the principles laid down 1n 


Chapter XIV and who bears in mind the nature of the groups and > 


the proportions of the unfits. 
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Comparison WITH RESULTS OBTAINED BY USE OF 
Oris Apvancep TrsT 


Miss Young has collected results for 469 pupils where we know 
both the Otis IO and the results of the Junior Leaving Certificate 
examination, "These are given below, and while they do not 
provide any statistical basis for the answer to our question, they 
may act as a warning to anyone who thinks of misusing our results 
by advising a pupil with an IQ of less than 100 to abandon the idea 
of taking a Junior Leaving Certificate course: 


IQ from Otis Number who Number who 
Advanced Test passed failed 
130-134. 3 xS 
125-129 26 
120-124 72 2 
115-119 117 7 
110-114 104 23 
105-109 61 21 
100-104 20 2 
95-99 5 4 
90-94 I 
85-89 I 
Total 410 59 


DAY SCHOOL CERTIFICATE (LOWER) 


By using the method of Chapter XIX we find that in the junior 
Secondary group a pupil with an IQ of 94 has an even chance of 
obtaining this Certificate. 


CHAPTER XXIV 
ATTAINMENT AND NECESSITY 


We have taken the title of this chapter from the admirable Report 
which the Glasgow Education Committee has published on М8 
examination held in May 1939; and we offer the following 
results as a small additional contribution to the study of this 
important social problem. . 1а 
No one who has handled the data of an Inquiry like ours yc 
fail to be struck by the varying grades of material with which f 2 
different Qualifying teachers have to work; nor could һе be di 4 
from doubt as to-whether full recognition is given to the splen x 
work being done by some of the teachers with classes of poor 3 
ability. We give below a comparison of two schools with differen 
qualities of Qualifying material: : 


Social ^ Average 


rating health Mean Mean 
of grading percentage 

school of pupils Qe Qa IO ЕО AQ? 
A 347 650 682 пбо 143 1135 
р * 2:95 322 248 89:6 897 07 


One feels that а comparison of this kind should be supplied to 
every newly appointed Inspector of Schools with the instruction 
that it should be prominently displayed on his desk when he 1s 
writing his reports. Indeed, it is difficult to believe that without 
the help of the scientific test results it would ever be possible to 
do justice to the work done by the Qualifying teacher in the 
second school. From the examination results we note that the 
average mark in English in the second school is less than half that 
in the first: in Arithmetic it is considerably less than half. "Тһе 
difficulty in realising that the teachers of the second school аге 
neither lazy nor inefficient is due largely to the difficulty of believ- 


1 Statistical Report of Examination held’ in Мау 1939. Corporation of 
Glasgow Education Department, ay 1939 orp 


2 We use the symbol AQ to denote Arithmetic Quotient, not Accomplish- 
ment Quotient, 
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ing that the mean IOs of two schools could differ by as much as 
26:4 points. Yet this is actually so; moreover, from the last two 
columns we see that the teachers in the second school are making 
more of their opportunities than those in the first. (If the teacher 
is making full use of her material the EQ and AQ should be the 
same as the IQ.) 


Socran RATINGS or THE SCHOOLS 


For some years before the Inquiry started we had been studying 
the relationship between intelligence and necessity, the investiga- 
tions being conducted by Miss Young and Dr McIntosh, assisted 
by Mr James P. Shepherd, М.А. For this purpose the schools were 
graded into four classes as regards necessity by the School Medical 
Officer, Dr A, E. Kidd, who brought to the problem a very inti- 
mate knowledge of the schools and of the districts from which 
they draw their pupils. He decided upon the grouping, after much 
experiment, because he felt that there was a clear gap between 
each class, that there was reasonable homogeneity within each 
class, and that in going over the list at different times he 
allocated the schools to the same classes without hesitation. 
Confirmation was obtained by submitting the lists to several 
others who were specially conversant with the social conditions of 
the City. The grouping was revised at the time of each investiga- 
tion, and a few alterations made in the light of changing social 
conditions. 


Levers or INTELLIGENCE IN THE DIFFERENT GROUPS OF 
Scuoors 


Five surveys of the intelligence of the schools at intervals since 
1931 have been made under .Miss Young's direction. On the 
Northumberland II Test the mean IQs of the different categories 
range from 117 to 102, while on the Simplex Junior Test they range 
from 109 to 83. We may therefore assume that the IQs obtained 
by the different tests are not comparable; and to facilitate com- 
parison we transformed the data in such a way as to make the 
mean IQ тоо and the standard deviation 15 for all tests. The 
mean IOs for the different groups of schools are given on p. 234. 

These figures reveal a striking fall in level of intelligence as we 
descend the scale of necessity, and we hope that they may do 
something to ensure that full recognition is given to the teachers 
in the poorer Areas who have to struggle year after year with 

ж 


а 
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difficulties not always appreciated by Education Committees 
That the results are worthy of confidence is shown by the remark- 
able consistency from year to year: 


1931 1932 1932 1933 1936 
Northumber- | Northumber- Mental pe MHT 21 
Social land I land II Survey (All (June 
rating (All Qualify- (All Qualify- (Age-group Senor Ur Qualifying 
ing classes) ing classes) 10}-114) casei) classes) 
A 14 111-9 111-7 111-8 113-1 
B 102-5 104-6 100-7 103-0 106-8 
c 98.9 98-5 98-9 99-8 99:5 
D 96-6 96-4 98-3 958 96-1 
VARIATION IN LEVEL ОЕ INTELLIGENCE IN INDIVIDUAL 
Scuoors 
Тһе avera, 


Bes given above may be steadied by weight of numbers, 


and it is interesting to find out whether the teacher in a D school 


© A SCHOOL 


,9 B.ScHOOL 


2 
P4 


INTELLIGENCE QUOTIENT 


i952 — —1953 1936 
OTTIS] SIMPLE: ats 
Suit онок мита 


Fic. 63.— Variation of intelligence level of three schools in different years. 
has a steady succession of 
occasional good year. 


the fluctuations from 
school about the mid 


poor classes, or whether she may have an 
The answer is given in fig. 63, which shows 
year to year for a good school in class А, а 
dle of class В, and a school low in class D. 
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The IQs have been made comparable according to the procedure 
described above. 

The variations are considerable and show that it would be 
unfair to judge the work of a Qualifying class even on the basis 
of a sound general grading of the school. In the D school the 
teacher of the Northumberland II group had a much better 
chance than her colleague who had the group taking MHT 21. 
There appears, in fact, to be no escape from the conclusion that 
no published assessment of the work of a teacher should ever be 
based on a general impression of the intelligence of the class or of 
the normal intelligence level of the school; scientific determina- 
tions of the intelligence and attainment levels of the individual 
class are essential. If this principle be confirmed, a revision of the 
System of school inspection would appear to be desirable. 


SCHOOL Аттлтхмехт IN THE DIFFERENT GROUPS OF Scuoors 


Тһе following data are for the June group only: 


Social Mean Nro 
rating percentage 
Qe Qa IQ EQ AQ 
A I 8- 1131 1138 111-8 
B же pe be 106-8 106-2 108-0 
С 468 420 995 997 10190 
р аға 33:6 96-1 95:6 971 


Both in English and Arithmetic we find the same steady falhas 
we descend the scale, and the drop from the highest class to t е 
lowest is very considerable, Yet the quotients show that in a 
Broups the teachers are making good use of their opportunities, 
Particularly in regard to Arithmetic. 


Comparison or THE GROUPS OF SCHOOLS IN REGARD TO 


HEALTH, INDUSTRY AND PRACTICAL ABILITY 


Only for health could we regard the gradings as strictly com- 


Parable, For the other criteria the differing standardo tt 


the teachers constitute a difficulty; yet it is one w \ 
tend to mask he Шера moni ың create them. In practical 
subjects the teacher of a poor class, having little aped 
Teally good classes, might tend to give a higher pope 

and B gradings than a uniform standard for the whole popu ation 


236 SELECTION FOR SECONDARY EDUCATION 


would justify. If, therefore, we find that there is evidence of а 
fall, it is, for this reason, all the more convincing. 


Miss Young made the following comparison by giving 5 points 
for an А, 4 for a B, and so on: 


Social rating Health Industry Practical subjects 


A 3:38 3:32 317 
B 317 317 323 
С 317 3:09 3:08 
D 316 2:98 3:05 


Such results have social implications which cannot be discussed 
here; but we present them in the hope that they may be of use 
to workers in this important part of the educational field. They 
suggest that the children of the better-class schools are not only 
more intelligent and of higher school attainment than those in 
districts where necessity is greatest; they are also healthier, more 
industrious and even of higher practicalability. "Тһе only devia- 
tion is that curious one in the last column, which indicates that 
the pupils in the B schools are better in practical ability than those 
in the A schools. This may be more than an accident. The B 
school pupils consist to a considerable extent of children of skilled 
artisans; the A schools are largely attended by children of pro- 
fessional classes. Speculation as to whether the difference is due 
to interest caught in the home, or to an atrophy of practical 
ability in the classes who have not to use their hands, may, however, 
wait until sounder investigations confirm its existence. ? 


СНАРТЕК ХХУ 
FIXING PASS-MARKS IN PRACTICE 


We have shown the importance of the correct fixing of pass-marks 
When examinations or batteries are used for selection, and we have 
hitherto determined these in terms of the standard deviation of 
the complete Qualifying group. We shall now show how they 
could be used in administrative practice; but before doing so, we 
shall consider briefly some other methods of fixing pass-marks. 


Can WE TRUST TO THE PRIMARY TEACHERS TO FIX THE 
Borper-Lines? 


We can make a rough estimate of the primary teachers’ pass- 
mark for admission to senior secondary schools as follows. For 
each pupil they gave a forecast of success on a five-point scale in 
which A and B indicated clear success, D and E certain failure; 
to the C category were relegated the doubtful cases. If we take 
the sum of the A and B gradings and add half of the C gradings, 
we get an estimate of the number of pupils who would fall above 
the pass-mark if it were drawn by the primary teachers. On this 
basis we find that they would pass about 36 per cent. of the whole 
Qualifying group; while the lowest of our scientifically determined 
border-lines (that is, that which would admit those who have the 
ability necessary for success, not merely those who would actually 
succeed) would pass 25:7 per cent. The primary teachers’ pass- 
mark for success in the senior secondary course is consequently 
too low, 

That the answer to the question- is a negative is confirmed 
When we study the forecasts of the individual schools. In one 
Schoo] the primary teacher thought that 29 out of a class of 40 
Would be certain successes in a senior secondary course; the 
external examiner thought that the number was 3; and, after 

aving the pupils for three years, the secondary head masters 
found that 3 was the correct number. 
237 


238 SELECTION FOR SECONDARY EDUCATION 


Can WE TRUST TO AN EXTERNAL EXAMINER TO FIX THE 
Pass-Mank? 


Аз far as our examiner is concerned the answer is an affirmative. 
The percentage of the whole group which falls above his i. 
mark was 25:4; for our scientifically determined border-line t 5 
percentage is 25-7. One examiner is, however, too small a samp 
to form a sound basis for generalisation, and we made the follow- 
ing further test. From the examination scripts of the Deren 
group Miss Young made up sets for English and Arithme 3 
separately, in which there was a graded series at intervals of I ре 
cent. from the best to the worst paper. These were submitted by 
Dr McIntosh to a number of other examiners who were asked to 
read them over and, accepting the marking system already used, 
to indicate the point at which they would fix the pass-mark for 
admission to senior secondary schools. The test which they were 
asked to apply was that a child should be admitted if he had more 
than a 50 per cent. chance of obtaining a minimum Senior Leaving 


Certificate in a suitable course in six years. The following were 
the pass-marks fixed: 


А Pass-mark Percentage of pupils 
Examiner per cent. admitted 
I 758 9:8 
2 72:0 14:7 
3 710 16:5 
4 68-0 21-7 
Š 5 67-0 23°5 
; 6 654 . 26:6 
7 64-4. 28-4 
8 558 467 
Inquiry examiner 68-0 217 
Correct pass-mark 66-8 23.9 


From these results we ma 
question is again a negative. 

бо far as the examination is concerned, considerable improve- 
ment could no doubt be effected if the examiner were kept in- 
formed of the later success or failure of the candidates, From the 
principles which we have laid down he could ascertain what the 
number of failures should be if his pass-mark were correct, and 
he could then keep a set of standard answer Paper. Yet question 


y conclude that the answer to our 
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papers vary in difficul 

деге var ty from year to year, and all i 

eo is far from easy. 4 os EE 

Even d 
es 5 die could rely upon the examiner to fix the pass-mark, 
5” е further complication that few Education Committees 
m ш we hope, use a single examination. The problem of 
ure is therefore that of fixing the pass-mark for a battery. 


Роѕзівпитү оғ ADMITTING А Frxep PERCENTAGE OF 
CANDIDATES TO SENIOR SECONDARY COURSES 


? mise year of our Inquiry 75 per cent. of the candidates for 
if the E to senior secondary courses would have been admitted 
Sred И orrect pass-mark had been used. With this percentage 

or the year 1935-1936 it might be thought that we could 


a it also in the year 1936-1937. We believe that this very 
ple system would work out just as fairly as most of the methods - 


ви pass-marks which аге ordinarily used; but it is unsatis- 

f y zs various cogent reasons. It is fixed on small numbers; 

bens ity of a small group of this kind may vary considerably 

diff year to year; and the percentage for one Area might be very 
erent from that for other Areas. È 


Tur Метнор RECOMMENDED 


. To Secure an objective standard арр 1 
1 a difficult problem, which may in the end be found insoluble. 

5 believe, however, that the following method would give a fair 
Preliminary approximation, and would be well worth а trial. It 
is based on the belief that the standard must be fixed in relation to 
а group so large and representative that it will vary little in quality 
and scatter from year to year. In City systems the complete 
Quility ing group would satisfy this condition; and we may 
urther assume that the quality of these large groups would not 
vary greatly from Area to Area. If the variations were consider- 
Шы the Director of Education could determine a small allowance 
п the light of his experience. 
f e have shown that in such a group the corre à 
Or the best battery for candidates desiring admission to Senior 
Secondary schools is +70 sigma. Now 479 times the standard 
мы ton of the whole Qualifying group represents 4 definite 
5 andard of intelligence and scholastic attainment, which will be 
teady from year to year and from Area to Area. We therefore 


correct pass-mark 


licable throughout Scotland 
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3 lifying 
ggest that the soundest method is to test the whole Qua fyin 
ou and fix the pass-mark at 44; of the standard deviation above 


the mean. We have given other reasons for the desirability of 
testing the complete group on p. Igo. 


PnacricAL PROCEDURES 


In this section we show how the above method would work ki 
in practice and how much clerical labour would be involved. o 
meet varying tastes, we begin with simple and inexpensive systems 


and end with the somewhat elaborate one which, in our opinion, 
would give the best results. 


A SIMPLE SYSTEM WHICH AVOIDS THE EVILS OF EXTERNAL 
EXAMINATIONS 


- This would use the battery Ї+Тї 


teachers estimates scaled on these) and 
would be as follows: 


(intelligence scores and 
the practical procedure 


Scale the teachers’ estimates in English + Arithmetic on the 
intelligence scores, 

Find the mean and standard de 
intelligence scores and teacher; 


The pass-marks will be the fo. 
mean: 


viation of the totals of the 
5” scaled marks. 


llowing distances above the 


(а) Candidates for admission to senior secondary schools, 
*66 times the standard deviation. 

(P) Bursary candidates, -93 times the standard deviation. 

(с) Candidates for admission to two-language courses, 
1:39 times the standard deviation, 


: а Clerkess іп an 
istician’s quick and 

Suppose that the mean of the 
deviation 30. The pass- O senior secondary 
schools is at once found t that is, 123, 


FIXING PASS-MARKS IN PRACTICE 241 


А SIMPLE SYSTEM WHICH INCLUDES AN EXTERNAL EXAMINATION 
The battery used in this case would be 1 +Q 4 Ts and the steps 


are as follows: 


Scale the teachers’ marks on the examination marks, taking 
English and Arithmetic together. 

Find the mean and standard deviation of the totals of 
1+Q+Ts. 

The pass-marks will be the following distances above the 
mean: 


(а) Candidates for admission to senior secondary schools, 
770 times the standard deviation. 

(b) Bursary candidates, -97 times the standard deviation. 

(с) Candidates for admission to two-language courses, 
1:43 times the standard deviation. 


„It is now hardly necessary to explain that this method allows for 
differences in the standard of difficulty of the examination papers 
from year to year. If the paper has been easy the pass-mark as 
Calculated above will be higher. 


THE SYSTEM WHICH WE REGARD AS THE BEST 


, This would be based on the battery I--Q 4- Ts, and would 
involve the following steps: 


Scale Te on Qe, giving Tes. 

Scale Та on Qa, giving Tas. 

Find the mean and standard deviation of the totals of I, Qe, 
Qa, Tes and Таз. 

Тһе pass-marks will then be determined as in the second 
System. 


Ап economical Committee could stop at that point. We 
Should, however, like to tempt all Committees to undertake 
the following further steps which, we can assure them, are not 
so laborious or time-consuming as they would seem: 

"Transform I, Qe, Qa, Tes, Tas and the average into standard 
Scores. 

Enter these on cards, as shown in fig. 64, the right-hand 
side of which would be designed to meet the tastes of the 
Committee. According to our view the spaces for ‘health,’ 
‘personal qualities’ and ‘home conditions’ would, in the 
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great majority of cases, be left blank. Entries would be 
made by the primary teacher only for applicants for ad- 
mission to senior secondary courses, and only when he felt 
that the pupil should be referred to the Medical Officer 
or to the staff of the Child Guidance Clinic. The right- 
hand side could also be adapted for follow-up records if 
that should be desired. 


+3 


+2 


SPACE FOR NAME, SCHOOL etc 
AND FOR ENTRIES IN REGARD TO 
HEALTH 
HOME CONDITIONS 
PERSONALITY 


-2 


Se Tes Qo Tas Ау. 


Fie. 64.—Qualifying record card. 


No system which uses ordinary percentages could ever compete 
with standard scores which have a clear, definite and constant 
significance. The lower pass-mark, shown by a thin unbroken 
line, is that for admission to senior secondary schools, "Тһе others, 
which also relate to senior secondary schools, are for bursaries and 
admission to two-language courses. 


With this system the following supplementary principles should 
^ be used in border-line cases: 


Refuse admission to all candidates wh 
the whole Quali 
z (Qa + Tas). 


: o fall below the mean of 
fying group in any one of I, (Qe + Tes) and 
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Admit pupils with average scores of -6 to -7 sigma if they are 
above 1-3 sigma in I, (Qe & Tes) or (Qa+Tas), and are 
not already failed under the first supplementary principle. 

If a pupil's Qe or Qa score is less than the corresponding 
teacher's scaled estimate by more than -7 sigma, such a 
score should be omitted in calculating the average. 


In fig. 64 the pupil has made a flop score in Arithmetic and 
would be an admit. He would not, however, be awarded a 
bursary; nor would he be admitted to a two-language course. 

Such cards, which can be interpreted at a glance, would be of 
considerable value to the heads of the secondary schools in advising 
pupils and parents and in making theirfirstclassification. | 


THE SYSTEM RECOMMENDED FOR RURAL AREAS 


In rural Areas we have recommended that the battery used 
should be I--Q. No scaling of teachers’ estimates is required, 
and the steps are as follows: 


Add the intelligence and examination scores and find the 
mean and standard deviation. " 
The pass-marks will be the following distances above the 
mean: 
(а) Candidates for admission to senior secondary schools, 
-66 times the standard deviation. 
(b) Bursary candidates, "93 times the standard deviation. 
(c) Candidates for admission to two-language courses, 
1:39 times the standard deviation. 


It should be borne in mind that the pass-marks suggested above 
are based on data for a single industrial City. In all Areas, but 
Particularly in rural Areas, small adaptations may have to be made 
in the light of experience and of knowledge of the social conditions. 


APPENDIX А 
QUALIFYING EXAMINATION 


DICTATION AND SPELLING 


Tuesday, 26th November 1935 
9.30 a.m. to 10.15 a.m. 


DICTATION 
InsTRUCTIONS 


The passage should be read over once before writing begins. , 
he passage should then be dictated according to the phrases indicated. 
i аралы should be dictated twice, and all punctuation should be 
ndicated, 


After the dictation is completed the passage should be again read over. 

These complaints | now became serious. | The admiral restrained 
them | by the calmness | of his countenance. | He called upon 
Heaven | to decide between himself | and the sailors. | He showed no 
fear, | Не offered his life | as a pledge, | if they would but trust 
him | and wait | for three days more. | He swore that, | if, | in the 
Course of the third day, | land was not visible | on the horizon, | he 
would yield | to their wishes | and steer for Europe. | The mutinous 
men | reluctantly consented | and allowed him | three days of grace. | 


SPELLING 


INSTRUCTIONS 

Each sentence should be read once and the class instructed to write 
own only the word underlined. The word may be repeated if necessary. 
- Britain is a member of the League of Nations. 

- The leader of the army was a very courageous man. 

- The school had a half-holiday on Wednesday. 

- On the top of the precipice there stood an old castle. 
- À grocer must be able to distinguish between different kinds of tea. 
- The mysterious disappearance of the visitor was the talk of the 

neighbourhood. 

7. Most new houses are lit by electricity. 

8. Please cut the cloth with a pair of scissors. 

9. The number of tramcars was not sufficient for the large crowd. 
I0. When his arm was bandaged, the injured man felt relieved. 
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„* 


QUALIFYING EXAMINATION 


ENGLISH—COMPOSITION 
Tuesday, 26th November 1935 


10.45 a.m. 


(Time Allowed—35 Minutes) 


Write a composition on ONE of the following subjects : 


has ^ т. Tell the story of one of the most interesting incidents in the life of 
Bonnie Prince Charlie or Mary, Queen of Scots. 


2. Imagine you have moved into a new house, Describe the house, 
indicating its situation and surroundings, its appearance from the 


outside, the number of rooms and what each is used for, the 
garden, etc, ы 


3. You have Spent a week’s holiday with your school friend. Write 
a letter of thanks and refer to some of your most pleasant 


experiences, 4 


24 The shops at Christmas time. 


ENGLISH—GENERAL 


Tuesday, 26th November 1935 
1.30 p.m. 


(Time Allowed—One Hour) 


* Ра 
“ 
" 2 
ж 
" 
А ы 
“ 
“ La 
+ * 
Ж 
* ^ " 
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TIME ALLOWED—ONE HOUR 


1. Read the following passage carefully and then answer the questions 
that follow. Write your answers in the spaces provided. 


In the course of long ages, the camel has become more and more 
suited to the country in which it lives, and to the work it has to do, 
- $0 that it is quite indispensable in the deserts. The feet of the 
camel are broad and flat, and instead of being protected by hoofs, 
they have expanded cushion-like soles enclosed in a hard skin 
which is not sensitive to the burning sand. This kind of foot 
makes it possible for a camel to walk over sand bearing a heavy 
load or a rider, where a horse would sink and flounder at every step. 
'The nostrils of the camel are slits which it can close at will to 
prevent the sand being blown into its breathing passages. 115 hump 
is a storehouse of fat, and when it has to go for a time with very 
scanty food, the fat is gradually absorbed to nourish the body. 
When the camel reaches good fresh grass, or any kind of green 
plant, the hump quickly swells again. 


(а) Name three parts of the camel's body which have become suited 
to its life in the desert. 


(с) How does the camel шапа 


fangen? ge to keep sand out of its breathing 


APPENDICES 249 
(^) What causes the hump to disappear? 


(b) Write down the word opposite in meaning to each of the 
following : 


expanded; sss sassarese die san serene  .. — a Dy 
possible..... se deg Sate vn ose ee кеа картаны eem 
gradually а.о Henn "i 
(0 Write down the noun corresponding to each of the following 
verbs : 
Еб еб га: ois keys des ves ха жәй жа M NITIDO түе 
ехрапа............ оен —Á 
DI MM nC 


(7) Write down the meaning of each of the following words : 


іп зрепваМе................... 2. без еее. 
бедейіте......«е»»-»» «еее аа баб Жара SN «өз ete ase 
absorbed. aes oa eeso ons dials коо гава Өз ее ase aci nain 


e 
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2. Read over the following sentences and then answer the questions 
-~ below: 
(1) When the meal is over we sit down round the camp fire. 
(2) The servants begin to clear away what is left. 
(3) The guide we took with us is on guard. 


| 
^ 


(2) Write down the subordinate clause in each of the three sentences 
and tell its kind and relation. 


@) Tell what part of speech each of the six words underlined is. 


(с) Маке six sentences in which each of these six words is used as a 
different part of speech : 


3. Rewrite the following passage correctly, using wi 
commas, periods, question marks, quotation marks, 


4. Combine each of the following sets of short sentences in 
sentence with the same meaning, making any necessa 
In doing so you may use any connecting word except ' 


т г. 
(а) The bus arrived. The children were all very excited. | 
taking them for a trip to the seaside. А 


(8) I received a letter this morning from ту uncle, He 
farmer. He invited me to spend my holidays with 
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QUALIFYING EXAMINATION 


ARITHMETIC—MENTAL 


Wednesday, 27th November 1935 


9 a.m. 


(Time Allowed—1o Minutes) 


I. Do not turn over this paper until you are told to do so. 


2. All working is to be done mentall 
are to be written down. T. UMEN 
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ARITHMETIC—MENTAL 


No marks will be allowed for answers that have been altered. 


Space For ANSWERS 


1. 34 +109 +28 4-397. ККУ ое 

2. Take 99 from 10,001. pa E 

3. Multiply 964. by 25. je ennt 

4. Find the total of 6/84, 3/9, 4/24, and 8/7. qom Hn 

5. How much is left out of 14/9 after spend- Дөн LLL 
ing 3/23 and 2/94? 

6. How many 144. stamps can be bought for б. ........-....... 
half a crown? 

7. Find the cost of 1 gross of pencils at 14. 7 0 -еееееее 
per pencil. 

8. If a cwt. of coal costs 1/6 what amount of 8. ......- m e 
coal can I get for 30/-? 

9. I leave home at 10.55 a.m., walking at д. enm 
3 miles per hour. How far have I . 
Бопе at 1.35 p.m.? 

10. To-day is27th November. On what day 10. s. ое ене ааа 


of the week did 3oth October fall? 


254 SELECTION FOR SECONDARY EDUCATION 


? += “a 


QUALIFYING EXAMINATION 


” ARITHMETIC— WRITTEN 
) Wednesday, 27th November 1935 
ж 9.25 алп. 


(Time Allowed—One Hour) 


Norz.—All the necessary working should be shown. 
' 


т. (а) Subtract £49 тоз 83d from £83 6s 4d. 
> @) Divide £263 5s 44d by Ly 


2. (а) Add together 7-324, 23:65, 101-008. 
(0) Multiply 36-289 by 1000. y 
3. A train travelling at 45 miles per hour covers the distance between 


two telegraph poles in six seconds. How many yards are the poles 
apart? 


4. The following price list was hanging in a shop window : 
Sugar, 23d per lb. 
Butter, 1/14 per Ib. 
Eggs, 1/2d per dozen. 
Tea, 2/6d per Ib. 
Paraffin, 1/2d per gallon. 
Candles, 1d each. 

Ham, 1/1od per Ib. 


Flour, 2/8d per stone. 
Soap, 4d per bar. 
Custard Powder, 8d per packet. 
Vinegar, 6d per pint. 
Pepper, 13 per oz. 
Fruit Salad, 1/34 per tin. 
Rice, 44d per Ib. 
Find out from the above price list the total cost of the following : 
2] stones Sugar. 
8 bars Soap. 
33 15. Tea. 
% gallon Vinegar. 
I$ gallons Paraffin. 
13 lbs. Pepper. 
- 8 packets Custard Powder, 
. 4 tins Fruit Salad. 
5. A silver collection was taken at a meeting. There we 
crowns, 37 shillings, 81 sixpences and 1 50 threepences, 
of 2 guineas was paid out of the collection for the rent o 
How much was left? : 


те 15 half- 
The sum 
f the hall, 
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FOLLOW-UP CARD 
SECONDARY SCHOOLS 


(Block Capitals, Surname First) 


......... LII 


+ If the pupil is still on the roll at the end of session 1938-1939: 


(а) Has pupil completed a 3-year course? 


(b) If Jf so, indicate by means of an X the one of the following cate- 


gories into which the pupil comes: 


(1) Passed the Day School Cert, (Higher) 


(2) Presented for Day School Cert. (Higher) 
but failed. 


(3) Presented for Day School Cert. (Higher) 
but result not known, £g., because 


returning to school. 


(4) Did not sit for Day School Cert. (Higher) 


because not up to necessary standard 


(5) Did not sit for Day School Cert, (Higher) 


because proposes to continue Leav, Cert, Course 
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` (©) Has pupil failed to complete a 3-year сошѕе?................ 
@ If so, 

(1) Was this due to retardation in first year? .............. 

(2) Was this due to retardation in second year? ............ 

уп. Level of Success in T Ы; g ME 

je matics Science French Latin Couse |2547 | 83 3 
А А А А А А 
B B B в B в в в 
с с с с с с с с 
D DD в р р р р 
E E E E E E E E 


IX. Are you satisfied that the course actually taken was the one most 


suited to the pupils abilities and educational needs?....+.++ 
X. If not, 


(а) Do you think the pupil was unfit for promotion to a post- 


primary course at the time he/she was promoted? . . . . .. .. ж. 


(E) In which other Secondary or Advanced Division course would 


the pupil’s needs have been best met? 
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Ability, differences between schools, 
54, 88, 232 
of younger and older children in 
Qualifying group, 25, 27 
specialised, 100, 145, 146, 160, 187 
Admit-fails, 130, 138, 139, 149, 141; 
173, 175. See Chapters XIII, 
103; XV, 143 
Age allowances, 8, 9, 21, 24, 74, 157 
effect on correlations, 24 
Age-groups, 2, 3, 9, 19, 22, 26 
Allowances, age, 8, 9, 21, 24, 74; 157 
health, 148 
home conditions, 154 
industry, 149, 155 
personal qualities, 152, 157 
scientific systems of estimating, 155 
teachers' forecasts, 151, 156 
Arithmetic, correlations with English, 
22 
Qualifying examination papers, 3; 
252 
teachers’ estimates, 4, 36, 37, 39 
tests, 3, 11 қ 
Attainment and interest, 30 
differences between school classes, 
.51, 52 
differences between schools, 35, 39; 
. 43» 53; 232 
influence of environmental condi- 
tions, 177, 178, 182, 184, 212, 232 
necessary for success in senior 
secondary course, 214. See also 
Pass-mark 
primary teachers’ estimates, 4. See 
also Teachers’ estimates 


Backward group, 7, 13, 203; 204, 209) 
, 210, 218, 225 К 
Bain, J. T., 34, 44) 55 
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Batteries. See Chapters VIII, 64; 
XI, 84; XVI, 163 
correct pass-marks for, 112, 240 
‚241, 243 
correlations, 68, 73, 74 75, 83 8 
163 
fictitious, 163 я 
final order of predictive value, 163 
numbers of misfits, 114, 143 
predictive value for senior and junior 
secondary courses, 75 
weighting, 68, 81, 163 
Best battery, 165, 241 
and bursary awards, 196 
correlations, 84, 163 
Bivariate surface, 55, 122 
Border-line. See also Pass-mark 
between backward and normal 
classes, 225 
. between success and failure in 
secondary course, 105 
Border-line cases. See Chapter ХУ, 
143 
admit-fails, 175 
reject-successes, 180 
special consideration of, 143, 242 
suitability of batteries for, 168 
Bursary examinations, 4, 125. See 
Chapter XIX, 188 > 


Ceiling effect, 188, 190 
Chief Medical Officer for Schools, 4; 


233 
Child Guidance Clinic, 179, 186, 242 
Choice of secondary course, 6, 19 
Choice of type of school, 198 
Correlation technique, limitations, 86 
Correlations between measures at 
Qualifying stage, 19 
combination of, 19, 54; 64 
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Correlations, correction for differences 
in means, 59, 61, 62 
correction for range, 20, 57, 65, 66, 
7574 
differences between courses, 79 
first- and second-year class marks, 
171 
maximum, with success to be ex- 
pected at Qualifying stage, 172 
order of predictive value based on, 
75, 115, 163 
predictive significance, 124, 127, 141 
Courses, need for additional, 187, 202, 
214, 219, 220 
suitability of, 104, 185, 186 
transferences, 174, 201 
two-language, 182, 187, 201, 202, 
206, 226, 240, 241, 243 Р 
Crabb, С. А., 29 


Day School Certificate (Lower), 223 
grading for success, 109 E 
IQ necessary for success, 231 
number obtaining, 224 
numbers capable of, 218 
pass-mark for, 224. 
results for various courses, 224 
December and June groups, 2, 3, 188 
correlations, 59, 61, 69, 70, 190 
difference in intelligence and school 
attainment, 6o 
Director of Education, 4 
Discrepancies in selection, 96, 184, 191. 
See also Misfits 
Dundee Education Committee, 4, 6 


English, correlations with Аг ипе 


tic, 

22 

Qualifying examination Papers, 3, 
245 


teachers’ estimates compared with 
Qualifying scores, 35 
tests, 3, 11 
English Examinatio: 
mittee, 16 
Environmental conditions, 97, 1 54, 
177, 182, 184, 195) 212, 220, 232 
Error of prediction, 23, 124 
Examiner, paid external, 6 
classification of Pupils, 13 
pass-mark fixed by, 238 


ns Enquiry Com- 


External examination, escape from, 45, 
169 : 
selection system avoiding, 240 


Failure. See also Admit-fails В 
percentage among those admitted 
to senior secondary courses, 141 
reasons for, 176, 177, 178, 179, 184, 
212 
Fitness. See Chapter XXI, 211 
for secondary education, 48, 104, 
123, 129, 211 
for two-language courses, 201, 206, 
226, 240, 241, 243 
Flexibility of secondary system, 170, 
186, 201 
Floor effect, 190 
Flop, 102, 147, 160, 243 
Follow-up, 2, 6 
“сагі, 7, 256 
Follow-up marks. 
secondary course 


See Success in 


Glasgow Education Committee, 232 
Gradings, 4, 13, 27, 29, 30, 38, 9o, 104, 
109, 148, 149, 151, 154, 205 


Halo effect, 172 
Health, allowance for, 148 
average scores for senior secondary, 
junior secondary and backward 
Broups, 204 
effect on success, 99, 176, 185 
in schools of different social ratings, 
232, 235 
Medical Officer's grading, 4, 9o, 99; 
148, 205 


Industry, 27 
allowance for, 149, 155 
average scores for senior secondary, 
junior secondary and backward 
groups, 204 
in schools of different social ratings, 
235 
relation to Success, 155 
Inspection of schools, 1, 232, 235 
Intelligence quotient (IQ) 
average for different junior second- 
ary courses, 206 
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Intelligence quotient (IQ) 
average for different senior secondary 
Courses, 206 
average for senior secondary, junior 
secondary and backward groups, 
204 
comparison with intelligence score 
(1), 20, 73 
correlations, 64, 73, 84, 163, 190 
for Day School Certificate (Lower), 
231 
for Junior Leaving Certificate, 230 
for Senior Leaving Certificate, 229 
in schools of different social ratings, 
232 
rise in predictive value in later years 
of course, 77 
teachers' estimates scaled on, 45, 169 
Intelligence tests, 3, 9 
correlations, 19, 64, 73, 84, 163, 190 
Interests, 29, 185, 204 


Johnstone, D. С., 229 
Junior Leaving Certificate 
grading for success, 100 
IQ necessary for success, 230 
misfits for complete battery with 
Correct pass-mark, 119 
number obtaining, 221 
numbers capable of, 218, 222 
pass-mark for, 118 
results for various courses, 222 
Junior secondary courses 
correlations, 68, 163 
number entering, 7 
number fit for, 13, 218 
predictive value of measures and 
batteries, 72, 75, 163 
Junior secondary group 
average IQ, 204. 
distribution, 118, 203 
fitness for various levels of education, 
218 
health, 204. 
industry, 204 
interests, 204 
IQs and Qualifying scores for various 
courses, 206 
Qualifying scores, 203 


Kidd, J. C., 39, 83 


McIntosh, D. M., 14, 25, 26, 82, 190, 
233, 238 
Marking, standards of, 14, 16, 34, 49, 
54, 78 3 
Medical examination, 186, 242 
Mental Survey, 1932, 8, 234 
Minimum, imposition of, in separate 
tests and examinations, 143, 160, 
242 
Misfits. See Chapters XIII, 103; 
XV, 143; XVIII, 175 
actual and theoretical, 114, 117, 119 
boys outnumber girls, 18700 
bursary examinations, 191, 194 
composition, 106, 108, 111, 114, 118, 
138 
жж corresponding to actual 
numbers, 117 
curves, 132 
determination of minimum for test 
or battery, 110, 129 
effect of allowances, 148 
effect of applying supplementary 
principles, 143 
effect of raising pass-mark above 
correct point, 133 
minimum to be expected at Qualify- 
ing stage, 172 
minimum when pass-mark is fixed so 
as to admit pupils with more than 
a 5o per cent. chance of success, 
131 
atten and composition in Junior 
Leaving Certificate, 119 
numbers for tests and batteries, 114 
order of predictive value based on, 
її 
Thee giving minimum for each 
battery, 112 
range of entrance attainment, 107 
use of profiles to determine numbers, 
10 
value of Qualifying examination 
tested by number of, 105 
Mitchell, M., 82, 200 
Moray House Tests, 3, 8, 234 


Northumberland Tests, 233 


Omnibus school, 202 ( 
Otis Advanced Test of Intelligence, 22 
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Partial correlation methods, 25 
Pass-mark. See Chapters XIII, 103; 
ХІУ, 120; XIX, 188; XXV, 237 
correct, 108, 110, 112, 129, 131, 136, 
137; 195; 227; 240 и 
effect of raising and lowering, 108, 
130, 131, 133, 134. 
external examiner's, 238 
for high ability in one subject, 145, 
160 
for Junior Leaving Certificate, 118 
for two-language courses, 227 
in bursary examinations, 195, 240 
primary teachers, 237 —— 
recommendation, 239 
spread of candidates in bursary 
examination, 189 | J 
Personal qualities, 152, 157, 182, 185 
Plan of Inquiry, 2  . - 
Practical ability, 205, 235 
Predicting success of an individual 
pupil, 123 
Predictive value of batteries for senior 
and junior secondary courses, 75, 
163 
Predictive value of marks in first year 
of secondary course, 171 
Predictive value of measures, fall in 
later years of secondary course, 
76 
final order, 163 
order based on correlations and on 
misfits, 115 " 
order for junior secondary schools, 
72, 163 
order for senior secondary schools, 
67, 163 
significance of differences in final 
order, 166 
variation in school courses, 79 
Primary teachers, border-line fixed by, 
237 


estimates, 4, 6, 23, 27, 29, 34, 149, 
168 


forecasts of success, 87, 151, 237 
reports, 4, 186, 242 
Probationary period, 171 
Profiles. See Chapters XII, 86; XIII, 
103 
samples, 91, 176, 193, 211 
Promotion system, 6 


Qualifying data, calculation of average 
of tests and examinations, 91 
correlations, 19 
primary teachers’ estimates, 4, 6, 23, 
27, 29, 34, 149, 168 
rating cards, 4, 255 
reliability coefficients, 14 
standard errors of measurement, 15 
Qualifying examinations, 3 
correlations with bursary examina- 
tion, 190 
intercorrelations, 18 
papers, 245 
scatter of candidates compared with 
bursary examination, 189 
teachers’ estimates scaled on results, 
40 
value in selection for senior second- 
ary education tested by misfits, 
105 
Qualifying group, 2. 
ХХ, 197 -` 
comparison of boys and girls, 197 
correlations, 19 
desirability of testing whole, 42, 
190, 239 
survey, 197 
Qualifying record card, 242 
Qualifyingscore(IQ + Q + S + Ts), 197 
average for senior secondary, junior 
secondary, backward and retained 
groups, 202 
average for senior and junior second- 
„АТУ courses, 198, 206 
distribution for junior secondary 
courses, 207 
distribution for Qualifying group, 
‚197 s 
distribution for senior secondary, 
junior secondary, backward and 
retained groups, 203 
Qualifying stage, 1 
Questionnaire on choice of course, 179, 
180, 198, 201 
Questionnaire on pupils’ interests, 30 


See Chapter 


Range, correction for, in correlations, 
„2% 57; 65, 66, 71, 74 

Rating cards, 4, 27, 148, 255 

Recommendations, fixing of pass- 
marks, 239 
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Recommendations, follow-up tech- 
nique, 62 
retention, 208 
selection for secondary education, 
160, 168, 186, 239 
supplementary principles of selec- 
tion, 160, 168 
teachers’ estimates, 47 
Record Booklets, 7 
Refuse-successes in bursary examina- 
tions, 126, 191 
Reject-successes, 130, 139, 149, 173. 
See Chapters XIII, 103; XV, 143 
border-line cases, 180 
Qualifying scores, 180 
Reliability coefficients. 
ІП, 14 
Retained group, 
207 
Retention system and results, 207 
Robertson, A. 5., 83, 172 -~ 
Robertson, J. I. W., 27 
Robertson, R., 156, 198, 204, 206 
Rodgers, H. C., 179, 198 
Rural areas, system of selection for, 


22 160,243 


See Chapter 


6, 7 203 , 204, 


Scaling of teachers’ marks, 20, 23, 38, 
58, 190 
Scatter diagram, 120 
Scholarships, selection for, 125, 188 
School subjects 
pupils’ interests. See Interests 
pupils’ preferences, 31 
Scottish Education Department, 1, 
200, 223, 224 
Secondary education for all, 92, 219 
Selection for scholarships. See Bursary 
examinations 
general principles, 120 
problems, 1, 2, 123 
system avoiding external examina- 
tion, 45, 169, 240 
system for rural areas, 169, 243 
system including external examina- 
tion, 241 
system regarded as best, 241 
Senior Leaving Certificate, 48 
IQ necessary for success, 228 
numbers capable of, 207, 211, 218 


Senior secondary courses 

ability and attainment necessary for 
success, 215 

admission of fixed percentage, 239 

number entering, 7 

percentage fit for, 211 

percentage successful, 218 

potential and actual success, 211 

predictive value of batteries, 75, 163 

predictive value of measures, 67, 163 

provision of courses, 187, 202, 214; 
219, 220 

provision of places, 139, 220 

significance of different sizes of 
correlation in selection, 142 

value of Qualifying examination in 
selection tested by misfits, 105 

Senior secondary group 

average IQ, 204 

average Qualifying score, 204 

comparison of sections taking differ- 
ent courses, 205 

distribution of Qualifying scores, 203 

health, 204 

industry, 204 

interests, 204. 

numbers fit for various levels of 
secondary education, 218 1 

proportion capable of two-language 
course, 226 > 


Sex differences, effect of extraneous 


factors, 187 
industry, 28 
junior secondary group, 198 
misfits, 187 
pupils’ interests, 33 
Qualifying group, 197 
senior secondary group, 198 
Shepherd, J. P., 233 
Simplex Junior Group Test of Intelli- 
gence, 233 
Social rating of schools, 233 
Special classes for backward pupils, 6, 
7, 13, 207, 208, 209, 211, 218, 225 
Standard errors of measurement for 
tests and examinations, 15 
Standard scores, 87, 105, 242 
Standardisation of tests, 2, 8 
Stewart, H., 229 
Success in secondary course. See 
Chapters VI, 48; XIII, 103 
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Success in secondary course, best esti- 
mate of percentage, 215 
comparability of marks, 49, 88 
comparison with bursary awards, 193 
correlations for tests and batteries, 
163 
definition, 48 
50 per cent. chance, 91, 131, 215 
final assessment, 87, 104, 109 
maximum correlation to be expected 
at Qualifying stage, 172 
predicting, for individual pupils, 123 
_ primary teachers’ forecasts, 87, 151, 
237 
relation to industry, 15$ 
relation to personal qualities, 157 
relation to teachers’ forecasts, 156 
scaling of marks, 58 


Teachers’ estimates, See Chapter V, 

34 

attainment in primary school, 4, 6 
34 

industry. See Industry 

personal qualities. See Personal 
qualities 

pupils’ interests. See Interests 

scaling of, 20, 23, 38, 190 
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Tests. See also Predictive value of 
measures 
Arithmetic, 11 4 
comparison with results for English 
pupils, 9 
correlations, 19, 22 
English, 5, 11 
intelligence, 5, 9 
length, 3, 14, 163, 166 
unstandardised, 74. 
Thomson, G. H., 3, 9, 161 
Transference 
between courses, 174, 201 
between schools, 174, 200 
Two-language courses, 182, 187, 201, 
202, 240, 241, 243. See Chapter 
XXII, 226 


Variability of performance, 16, 100 Я 
Variance and co-variance within an 
among schools, 63 


Waste, 173, 211, 220, 223, 227 

Weighting of measures, 68, 163. See 
Chapter X, 81 

Wilkie, R., 49 
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